I direct the AI:Futures and Responsibility Programme (https://www.ai-far.org/) at the University of Cambridge, which works on AI strategy, safety and governance. I also work on global catastrophic risks with the Centre for the Study of Existential Risk and AI strategy/policy with the Centre for the Future of Intelligence.
Sean_o_h
Karma: 3,748
A quick note that there’s a session on it at this weekend’s EA Global: Reconnect (which Sanjay is speaking at): it might catalyse formation of such a group!
I haven’t done a systematic analysis, but at a quick glance I’d note that quite a number of the grants in scientific research seem like their outputs would directly support main EA cause areas such as biorisk and global health—e.g. in the last 1-2 years I see a number on malaria prevention, vaccine development, antivirals, disease diagnostics etc.
>but I think it’s worth noting that, were various longtermist ideas to enter mainstream discourse, this is exactly the kind of critique they’d receive (unfairly or not!) - so it’s worth considering how plausible these charges are, and how longtermists might respond.
This is a good point, and worth being mindful of as longtermism becomes more mainstream/widespread.
Addendum: There’s a saying that “no matter what side of an argument you’re on, you’ll always find someone on your side who you wish was on the other side”.
There is a seam running through Torres’s work that challenges xrisk/longtermism/EA on the grounds of the limitations of being led and formulated by a mostly elite, developed-world community.
Like many people in longtermism/xrisk, I think there is a valid concern here. xrisk/longtermism/EA all started in a combination of elite british universities + US communities e.g. bay. They had to start somewhere. I am of the view that they shouldn’t stay that way.
I think it’s valid to ask whether there are assumptions embedded within these frameworks at this stage that should be challenged, and to posit that these would be challenged most effectively by people with a very different background and perspective. I think it’s valid to argue that thinking, planning for, and efforts to shape the long-term future should not be driven by a community that is overwhelmingly from one particular background and that doesn’t draw on and incorporate the perspectives of a community that reflects more of global societies and cultures. Work by such a community would likely miss important values and considerations, might reflect founder-effect biases, and would lack legitimacy and buy-in when it came to implementation. I think it’s valid to expect it to engage with frameworks beyond utilitarianism, and I’m pleased to see GPI, The Precipice, amongst others do this.
As both xrisk and longtermism grow and mature, a core part of the project should be, in my view, and likely will be, expanding beyond this starting point. Such efforts are underway. They take a long time. And I would like to see people, both internal and external to the community, challenge the community on this where needed .
However, for someone on this side of the argument, I am deeply frustrated by Torres’s approach. It salts the earth for engagement with people who disagree with this view and actively works against finding common ground. It alienates people from diverse backgrounds outside xrisk/longtermism from engaging with xrisk/longtermism, and thus makes the project harder. And it strengthens the views of those who disagree with the case I’ve put, especially when they perceive those they disagree with acting in bad faith. The book ends with the claim “More than anything, I want this mini-book to help rehabilitate “longtermism,” and hence Existential Risk Studies.” I do not believe this hostile, polemical approach serves that aim; rather I worry that it is undermining it.
[disclaimer: I am co-Director at CSER. While much of what I will write intersects with professional responsibilities, it is primarily written from a personal perspective, as this is a deeply personal matter for me. Apologies in advance if that’s confusing, this is a distressing and difficult topic for me, and I may come back and edit. I may also delete my comment, for professional or personal/emotional reasons].
I am sympathetic to Halstead’s position here, and feel I need to write my own perspective. Clearly to the extent that CSER has—whether directly or indirectly—served to legitimise such attacks by Torres on colleagues in the field, I bear a portion of responsibility as someone in a leadership position. I do not feel it would be right or appropriate for me to speak for all colleagues, but I would like to emphasise that individually I do not, in any way, condone this conduct, and I apologise for it, and for any failings on my individual part that may have contributed.
My personal impression supports the case Halstead makes. Comments about my ‘whiteness’, and insinuations regarding my ‘real’ reasons for objecting to positions taken by Torres only came after I objected publicly to Torres’s characterisations of Halstead, Olle Hagstrom, Nick Beckstead, Toby Ord and others. I have been informed by Torres that I owe him an apology for not siding with him [edit: to emphasise, this is my personal subjective impression/interpretation based on communications with me].
As well as the personal motivation, this mode of engagement reflects another aspect of this discourse I find deeply troubling: while I think there are valid arguments against longtermism, and alternative perspectives, it becomes impossible to discuss the issues, and in particular, the unfair characterisation of individuals, on the object level. Object level disagreement is met with an insinuation that this is the white supremacists closing ranks. I do believe there is a valid argument in some cases that one can be unaware of biases, and one can be unconsciously influenced by the ‘background radiation’ of a privileged society. Personally I have experienced this in unconscious, and sometimes deliberate, racism experienced as an Irish person living in Britain, and I have no doubt that non-white people have it much worse. However, this principle can also most certainly be overused uncharitably, or even ‘weaponised’ to shut down constructive intellectual engagement. And it is profoundly anti-intellectual to permit only those from outside a system of privilege to challenge scholarship.
There are other rhetorical moves I find deeply troubling. The common-society use of ‘white supremacy’ is something like “people who believe that white people are superior to other races and should dominate them; and are willing to act on that through violent means.”. Torres has typically not defined the term, but when challenged, he has explained that he is using the term in the more narrowly-used way used in critical race theory; of “of white people benefiting from and maintaining a system where the legacy of colonial privilege is maintained”. (note that he does define it in the mini-book, although as the ‘academic’ definition, which I think is overstatement). When challenged, Torres insults people for not automatically knowing he is using the more esoteric CRT definition rather than the common-use definition. This is not a reasonable position to take. And it is not reasonable to expect people not to be deeply hurt and offended by the language used.
Even accounting for the CRT definition, this is still an extremely serious and harmful accusation, and one that should not be made without extremely careful consideration and very strong evidence. In my own case, as someone from a culture overwhelmingly defined by the harms of colonialism, it is another way of shutting down any possible discussion; it is so violently upsetting that it renders me incapable of continuing to engage.
To the extent that scholars at CSER are still collaborating with Torres: I am not. I have spoken regarding my concerns to those who have let me know they are still collaborating with him, and have let them make their own choices. Most collaborations are the legacy of projects initiated during his visit 2 years ago (which I authorised, not knowing some of the more serious issue Halstead raises, but being aware of some more minor concerns). Papers take a long time to go through the academic system, and it would be a very unusual and hostile step to e.g. take an author’s name off a paper against their wishes. In some instances, people wished to engage with some aspects of Torres’ critique and collaborate with presenting them in a more constructive and less polemical way (e.g. see several examples of Beard+Torres). I have respected their choices. This may not be the case with all collaborations; at CSER’s current size I am not always aware of every paper being written. But I think it is fair to say my view on this style of engagement are well-known.
I have not taken the step of banning colleagues at CSER from collaborating with Torres. This would be an extremely unusual step in academia, running contrary to some fundamental principles of academic freedom. Further, I am concerned that such steps would reinforce another set of attack lines: Torres has already publicly claimed that he ‘has no doubt’ that employees at CSER that disagreed with me would be fired for it. I value having scope for intellectual disagreement greatly, and I would not want this perspective to take hold.
I do not claim that my decisions have been correct.
I do think there is significant value in engaging with critics. I admire engagement of the sort that Haydn has just undertaken. As a committed longtermist, to ‘turn the other cheek’ and engage in good faith with a steelmanned, charitable interpretation of a polemical and hostile document is something I find admirable in itself. And as noted elsewhere in this discussion, enough people have found some value in the challenge Torres has presented to ideas within longtermism (even where presented uncharitably) that it seems reasonable for some to engage with it. However at the same time, I do worry that beyond some point, engaging so charitably may legitimise a mode of discourse that I find distressingly hostile and inimical to kind and constructive, and open discourse.
These are challenging, and sometimes controversial topics. There will very often be issues on which reasonable people will disagree. There will sometimes be positions taken that others will be profoundly uncomfortable with. This is not unique to Xrisk or longtermism; the same is true of global development and animal rights. I believe it is of paramount importance that we be able to interact with each other as thinkers and doers in a kind, constructive and charitable way; and above all to adopt these principles when we critique each other. After all, when we are wrong, this is nearly always the most effective way to change minds. While not everyone will agree with me on this, this is the view I have always put forward in the centres I have been a part of.
A quick point of clarification that Phil Torres was never staff at CSER; he was a visitor for a couple of months a few years ago. He has unfortunately misrepresented himself as working at CSER on various media (unclear if deliberate or not). (And FWIW he has made similar allusions, albeit thinly veiled, about me).
Very helpful, thanks. Separately, I’d note that several of the biosecurity researchers most actively getting involved with the GCR community (e.g. Piers Millett at FHI and Catherine Rhodes at CSER) did PhD/postdoc work with Malcolm. (I’ve also found him a lovely and approachable person, generous with guidance and feedback—we’ve had him at several workshops).
For context, the specific ‘question about the ethics of political violence’ was itself somewhat inflammatory:
”So you’re in favor of mob violence, as long as it comes from the left?”
https://twitter.com/pmddomingos/status/1346940377840848898
Found this very helpful, thank you!
There’s a powerful poem in my native language (Irish) that was published in 1971, whose title loosely translates to “Indifference cannot be permitted”. It calls for equality, compassion, and our obligation towards people in all parts of the world, people with mental illness, non-human animals, and (depending on how one translates) possible life beyond earth. It was my first introduction to principles such as those that underpin EA. I won’t try to translate it, but it’s talked about (and part of it translated) in a recent blog post here: https://www.ria.ie/news/membership-policy-and-international-relations/ni-ceadmhach-neamhshuim
Níl cuil, níl leamhan, níl beach
Dar chruthaigh Dia, níl fear,
Nach dualgas dúinn a leas,
Níl bean; ní ceadmhach neamhshuim
A dhéanamh dá n-imní;
Níl gealt i ngleann na ngealt,
Nár chuí dhúinn suí lena ais,
Á thionlacan an fhaid
A iompraíonn thar ár gceann
Ár dtinneas-ne ’na mheabhair.
Níl alt, níl sruth, níl sceach,
Dá iargúlta iad, níl leac,
Bídís thuaidh, thoir, thiar nó theas,
Nár cheart dúinn machnamh ar a suíomh
Le gean is le báidhíocht;
Dá fhaid uainn Afraic Theas,
Dá airde í gealach,
Is cuid dínn iad ó cheart:
Níl áit ar fuaid na cruinne
Nach ann a saolaíodh sinne.
A couple of resources that may be of interest here:
- The work of Aviv Ovadya of the Thoughtful Technology Project; don’t think he’s an EA (he may be, but it hasn’t come up in my discussions with him): https://aviv.me/
- CSER’s recent report with Alan Turing Institute and DSTL, which isn’t specific to AI and social media algorithms only, but addresses these and other issues in crisis response:
”Tackling threats to informed decisionmaking in democratic societies”
https://www.turing.ac.uk/sites/default/files/2020-10/epistemic-security-report_final.pdf
- Recommendations for reducing malicious use of machine learning in synthetic media (Thoughtful Technology Project’s Aviv Ovadya and CFI’s Jess Whittlestone)
https://arxiv.org/pdf/1907.11274.pdf
- And a short review of some recent research on online targeting harms by CFI researchers
Well done on this, important work and a strong set of signatories.
Yes, and this is likely to reflect that (1) initial outbreaks were concentrated in cities/international hubs, more of which vote democrat and (2) in the initial outbreak testing capacity was lower, meaning that there were likely many more undiagnosed cases in these states. Treatment and therefore survival has improved too, but I think overall Linch’s suggestion of mortality is a fairer metric for covid prevalence.
A quick note on epistemic security: we’ve just published a report exploring some of these ideas (previously discussed with GovAI) in partnership with the Alan Turing Institute and the UK’s Defence Science Technology Laboratory, and building on a previous series of workshops (which Eric Drexler among others participated in). For those interested, it’s available below.
“Access to reliable information is crucial to the ability of a democratic society to coordinate effective collective action when responding to a crisis, like a global pandemic, or complex challenge like climate change. Through a series of workshops we developed and analysed a set of hypothetical yet plausible crisis scenarios to explore how technologically exacerbated external threats and internal vulnerabilities to a society’s epistemic security – its ability to reliably avert threats to the processes by which reliable information is produced, distributed, and assessed within the society – can be mitigated in order to facilitate timely decision-making and collective action in democratic societies.
Overall we observed that preserving a democratic society’s epistemic security is a complex effort that sits at the interface of many knowledge domains, theoretical perspectives, value systems, and institutional responsibilities, and we developed a series of recommendations to highlight areas where additional research and resources will likely have a significant impact on improving epistemic security in democratic societies”
As a datapoint, the issues Ozzie raises feel quite relevant to issues I find myself needing to think about where it comes to different communities engaging with Xrisk-related issues and different aspects of our (an xrisk/gcr centre’s) work—especially when it comes to different communities with different epistemic and communication norms—so I find it relevant and helpful in that sense.
+1!
Thanks Juan, I hadn’t seen that most recent R0 estimate you link to—concerning.
I’m not convinced that we would have already seen a significant uptick in reported/confirmed case numbers quite yet (weren’t the largest protests this past Saturday?). The median incubation period is ~5 days, most people don’t get tested at the time of symptom onset, and the PCR test turnaround time still seems to still be at least a day or two. Perhaps most importantly, most of the protestors seem to be relatively young and so many may be asymptomatic or may have mild cold/flu-like symptoms. I’m more interested in (and concerned about) any secondary transmission events that may involve older family members that protestors might live with/come into contact with. Many of these older folks would presumably have more serious symptoms and so would be more likely to show up in confirmed cases/hospitalizations data over the coming weeks.
Right. But with regard to R0 =0.9, I understand R0=0.9 was being used as the background R0 prior to the impact of the protest, rather than the R0 following the impact of the protests (if ‘background’ R0 is <1, then the impact of an R0-increasing event/set of events will have a lesser effect than if ‘background’ R0 is >1). It may be the case, as you suggest, that R0 has increased significantly since the start of the protests until now (whether due to the protests or in combination of other factors), in which case protests right now are happening against a higher R0 than these estimates assume—but we don’t have the data. I agree that NYC will be interesting.
R0 could certainly be much higher in principle, though if it is, it doesn’t seem to be reflected in the number of positive cases being recorded in the US—which has been holding steady or slightly declining for the past month—or the number of deaths (declining, although there would be a lag there). These indicators could be misleading of course—the US, like the UK, is nearly certainly undertesting and undercounting cases. However, the number of tests has been going up, and if the number of cases being ‘caught’ isn’t increasing this is some indication that R0 is somewhere a little below 1. So I would tentatively agree with the OP’s suggestion here.
https://www.worldometers.info/coronavirus/country/us/
One reason looking at different assessments based on R0=0.9 is informative is that it illustrates just how high the uncertainty and room for error is in these sorts of analysis. The OP predicts 75,000 deaths as a plausible possibility; Bedford predicts 150-600/day, so 2,100 −8,400 over the course of 2 weeks of protests (assuming each day carries the same impact on R0, which is probably wrong); both using R0=0.9 as a central assumption, and recognising that the present R0 is a key factor. Because present value of R0 is such a critical factor, comparing different estimates at the same R0 makes it easier to compare/contrast.
One of my personal concerns is that the BLM protests may end up unduly scapegoated (in terms of their role being overestimated) for any increase in cases and deaths; the US administration has done quite a bit of scapegoating already in my view, and there are many ways in which its own response has been far from adequate. My intuition is that other aspects of states reopening prematurely are likely to play a bigger impact in a possible second wave. If there were a significant overestimation of the impact of the BLM protests for example, this would be bad not only for the BLM movement and antiracism in the US; it would also be bad in terms of understanding the other causes of increase of R0 and putting in the appropriate planning overall for preventing and responding to future waves. (Likewise however, if those of us who think the impact of the protests is lower than in reality are wrong, it would be good to update).
Happy to have a go; the “in/out of context” is a large part of the problem here. (Note that I don’t think I agree with Beckstead’s argument for reasons given towards the end).
(1) The thesis (198 pages of it!) is about shaping the far future, and operates on staggering timescales. Some of it like this quote is written in the first person, which has the effect of putting it in the present-day context, but these are at their heart philosophical arguments abstracted from time and space. This is a thing philosophers do.
If I were to apply the argument to the 12th century world, I might claim that saving a person in what is now modern day Turkey would have greater ripple effects than saving a person in war-ravaged Britain. The former was lightyears further ahead in science and technology, chock full of incredible muslim scholar-engineers like Al Jazari (seriously; read about this guy). I might be wrong of course; the future is unpredictable and these ripples might be wiped out in the next century by a Mongol Horde (as for the most part did happen); but wrong on different grounds.
And earlier in the thesis Beckstead provides a whole heap of caveats (in addition to ‘all other things being equal’, including that his argument explicitly does not address issues “such as whose responsibility that is, how much the current generation should be required to sacrifice for the sake of future generations, how shaping the far future stacks up against special obligations or issues of justice”; these are all “good questions” but out of scope.)
If Beckstead further developed the ‘it is better to save lives in rich countries’ argument in the thesis, explicitly embedding it within the modern context and making practical recommendations that would exacerbate the legacy of harm of postcolonial inequality, then Torres might have a point. He does not. It’s a paragraph on one page of a 198 page PhD thesis. Reading the paragraph in the context of the overall thesis gives a very different impression than the deliberately leading context that Torres places the paragraph in.
(2) Now consider the further claims that Torres has repeatedly made—that this paragraph taints the entire field in white supremacy; and that any person or organisation who praised the thesis is endorsing white supremacy. This is an even more extreme version of the same set of moves. I have found nothing—nothing -anywhere in the EA or longtermist literature building on and progressing this argument.
(3) The same can be seen, but in a more extreme fashion, for the Mogensen paper. Again, an abstract philosophical argument. Here Mogensen (in a very simplified version) observes that over three dimensions—the world—total utilitarianism says you should spread your resources over all people in that space. But if you introduce a 4th dimension—time, then the same axiology says you should spread your resources over space and time, and the majority of that obligation lies in the future. It’s an abstract philosophical argument. Torres reads in white supremacy, and invites the reader to read in white supremacy.
(4) The problem here is that no body of scholarship can realistically withstand this level of hostile scrutiny and leading analysis. And no field can realistically withstand the level of hostile analysis where one paragraph in a PhD thesis taken out of context is used to damn an entire field. I don’t think I personally agree with the argument on its own terms—it’s hard to prove definitively but I would have a concern that inequality has often been argued to be a driver of systemic instability, and that if so, any intervention that increases inequality might contribute to negative ‘ripple effects’ regardless of what countries were rich and poor at a given time. And I think the paragraph itself could reasonably be characterised as ‘thoughtless’, given the author is a white western person writing in C21, even if the argument is not explicitly in this context.
However the extreme criticism presented in Torres’s piece stands in stark contrast to the much more serious racism that goes unchallenged in so much of scholarship and modern life. Any good-faith actor will in the first instance pursue these, rather than reading the worst ills possible into a paragraph of a PhD thesis. I’ve run out of time, but will illustrate this shortly with a prominent example of what I consider to be much more significant racism from Torres’s own work.