I was disappointed GiveDirectly wasn’t mentioned given that seems to be more what he would favour. The closing anecdote about the surfer-philosopher donating money to Bali seems like a proto-GiveDirectly approach but presumably a lot less efficient without the infrastructure to do it at scale.
OscarD
Thanks for writing this! I agree making this decision very carefully seems well warranted. One minor thing—you generously offer for people to contact you to ask about out-of-scope things (delivery research). I can well imagine someone finding this post in several years’ time and wanting to contact you, but I’m not sure if this anonymous account will exist/be monitored then; one possible solution is to link to a google form that would go to your email (without revealing what your name/email is).
Thanks for this!
I think it would have been interesting if you had written out some predictions beforehand to compare to the actual lessons. I should have done this myself too perhaps, as in hindsight it is now easy for me to think that it is a priori straightforward that eg the size of the future (time and expansion rate) dominate the EV calculation for x-risk interventions. I think a key value of a model like this could be to compare it to our intuitions/considered judgements to try to work out where they and the model disagree, and how we can change one or the other accordingly.
I am also confused as to why we need/want monte carlo simulations in the first place. My understanding of the model is that cost-effectiveness is essentially the product of several random variables: cost-effectiveness = X * Y * (1/Z) where X~Lognormal(a,b), Y ~ Normal(c,d), Z ~ Beta(e,f). In this case can’t we just analytically compute the exact final probability distribution? I am a bit rusty on the integration required, but in principle it seems quite doable (even if we need to integrate numerically rather than exactly), and like this would give more accurate and perhaps faster results. Am I missing something, why wouldn’t my approach work?
In a separate comment I describe lots of minor quibbles and possible errors.
This analysis seems fair to me. One mitigating feature I think is that precisely because the vacuous applause light statements aren’t meant to be action-guiding, they generally are not action-guiding. Hopefully Bob makes his remark and then people nod wisely and go on with selecting the best candidates as if ~nothing had happened. I think there is still a danger that something that should have been a mere applause light is misinterpreted as action-guiding (as Alice did here) but this time by Chloe who accepts the remark uncritically. Chloe may then go on to form real policies and plans based on Bob’s remark. So yes this seems bad, but hopefully not really bad. In terms of what to do about it, perhaps if more people start engaging critically with applause light statements and they are shown up to be without much thought or substance, this renders making such statements negative in terms of social value and fixes the problem. But perhaps more likely is that the people engaging critically with applause lights are hounded down for being insensitive. Tricky.
I appreciate your willingness to change your mind and make this difficult decision. I think this is a big part of what makes EA great, thank you. (I make no comment on whether this is the correct decision at the object level, I don’t really know enough to say.)
Record time to safe clinical development
we believe this success is a testament to what’s achievable when a talented, altruistic group of people come together to tackle a seemingly impossible task
Amazing that you got to a clinical trial so quickly! I find myself confused as to how this could happen though: it would be quite surprising to me if other companies and people are just insufficiently clever or hard-working enough or something. Surely there is huge optimisaiton pressure that is already being brought to bear on making R&D and clinical trials go quickly and efficiently as this is a big chunk of how pharma companies make money, and so it seems strange if there were just lots of big wins waiting to be taken in improving the process. What do you think?
Thanks, useful thoughts, I think I roughly agree with you and will change this. I suppose the tradeoff I was facing with the title (not that I spent any time weighing up different options consciously) is between brevity, accurateness, and interestingness. I think the more complete title would be something like ‘Updating weakly against the Biological Weapons Convention being as important to work on as I thought’. I think I will change the title to ‘Reflections on the BWC’ so that people who only see the title don’t get a negative vibe (I agree we want people overall to think good thoughts about the BWC). And then if people are interested enough to read the post, they will see that I raise, quite sloppily/intuitively, some drawbacks. More than me arguing the BWC is −10 on some scale of goodness, what I was thinking is it moved from +20 to +10 or something.
I haven’t thought about it lots but I think I would endorse something like ‘the BWC should continue to exist, and should be larger and bigger and better, but it is less of a central priority than I thought, and so people who care about prioritisation and don’t have individual reasons that the BWC is unusually good for them should strongly consider focusing more on something else’.
Good on you for being courageous and scout-minded enough to shut this down (and to start it in the first place)! I hope you find great projects to move onto.
I think I am a lot more on board with promoting idea pluralism (I realise I should have said this in my original comment, I was focusing there on what I found more controversial or difficult to think about well). I think science generally would go faster if funders took more risks on heterodox ideas (particularly given most research projects have far larger upside risks than downside risks, so ‘hits-based’ funding could work well). That’s a good point re things being cheaper to run in poorer countries, so more cost-effective all else equal.
I can’t imagine the applicants are any less good
At one level, yes intelligence and creativity are ~evenly distributed worldwide. But I think this gets to my earlier point about educational and other opportunities currently being very unequally distributed, so I think it would be the case (unfortunately) that applicants with access to loads of opportunities to develop their thinking and writing and research skills, disproportionately in the rich world, will be better able to contribute straight away. I think there could also be a strong case to run such fellowships elsewhere with fellows who have had fewer opportunities and are currently less capable, as this is more additional, but this seems like a notably different theory of change.
To red-team a strawman of your (simulated) argument: what about the Pascallian and fanatical implications across evidentially cooperating large worlds? I think we need some Bayesian, anthropic reasoning, lots of squiggle notebooks, and perhaps a cross-cause cost-effectiveness model to get to the bottom of this!
Several (hopefully) minor issues:
I consistently get an error message when I try to set the CI to 50% in the OpenPhil bar (and the URL is crazy long!)
Why do we have probability distributions over values that are themselves probabilities? I feel like this still just boils down to a single probability in the end.
Why do we sometimes use $/DALY and sometimes DALYs/$? It seems unnecessarily confusing. Eg:
If you really want both maybe have a button users can toggle? Otherwise just sticking with one seems best.“Three days of suffering represented here is the equivalent of three days of such suffering as to render life not worth living.”
OK, but what if life is worse than 0, surely we need a way to represent this as well? My vague memory from the moral weights series was that you assumed valence is symmetric about 0, so perhaps the more sensible unit would be the negative of the value of a fully content life.“The intervention is assumed to produce between 160 and 3.6K suffering-years per dollar (unweighted) condition on chickens being sentient.” This input seems unhelpfully coarse-grained, as it seems to hide a lot of the interesting steps and doesn’t tell me anything about how these numbers are estimated, and it is not the sort of thing I can intelligently just choose my own numbers for. Also it should be conditional
In the small-scale biorisk project, I never seem to get more than about 1000 DALYs per $1000, even when I crank expansion speed to 0.9c and length of future to 1e8, and the annual extinction risk in era 4 to 1e-8. Why is this? Yes 150,000 is too few, but I thought I should at least see some large effect when I change key parameters by several OOMs. Not really sure what is going on here, I’ll be interested if you replicate this, and whether there is a bug or I am just misunderstanding something.
Thanks for sharing, it sucks that you went through this (and sucks that the moths went through this :( ). As uncomfortable as thinking about these topics is, I am glad to be part of a community of people who take ethics seriously and try to act with compassion and consideration. Let’s hope market forces take effect and enough people inquiring about low-suffering ways to kill insects creates a market for companies to offer this :)
Thanks Gideon and others for creating this, I’m glad it exists. I have only skimmed the other comments, so apologies if the following have already been discussed elsewhere. In my mind it is useful to separate two quite different sentiments, I’d be interested if others agree with/find useful this distinction:
We would prefer a world where demographic markers (perhaps most notably nationality) are not predictive of ability to participate in intellectual communities and knowledge-making.
Given the current very unequal and unjust world, we would prefer to fund/hire/support/etc people who do worse in our evaluation metrics (interviews, work trials, research proposals, etc) but are from more underrepresented backgrounds.
Both are in favour of greater pluralism and diversity. I think 1 is trivially true—of course a more equal and just world would be better (though it would be a further claim that this is something X-risk orgs should focus on). 2 I am more torn about, I think I more disagree than agree though. I think if two people, one from a very intellectually privileged background, and one less so, do equally well on our evaluation metrics, the latter person is likely to be more intelligent/creative/novel as they had a harder road to there and needed to do more for themselves. I also think though that it would be pretty bad (and often clearly illegal) for decision-makers to use demographic markers over and above evaluation metrics.
I think the “In Defense of Merit in Science” paper is relevant here, though I haven’t read more than the abstract and the public discussions of it. I am glad the statement talks about tradeoffs. I think this is another important tradeoff to add: the more dimensions you are optimising over, the less well you can optimise each of them. This seems uncontroversially true. The implication of this is that if we add more demographic dimensions to optimise over this will have a cost on whatever other dimensions we may already be using (novelty of ideas, clarity of writing, mathematical ability, etc). I think for me this is the most important tradeoff.
Nathan (or anyone else) I doubt I will read the whole google doc but would appreciate a ~1 page summary, and I think that would be appropriate to include in this post (with, of course, a disclaimer that it is your gloss on what Owen is saying)
Thanks Sarah, important issues. I basically agree with Aron’s comment and am interested in your thoughts on the various points raised there.
Like Aron, I think the route from a single nuclear deployment to an existential catastrophe runs mainly through nuclear escalation. I think you are arguing for a more expansive conclusion, but the smaller claim that the best way to prevent a large nuclear exchange is to prevent the first deployment seems very defensible to me, and is still big if true. I don’t have the necessary object-level knowledge to judge how true it is. So I agree with P1.
For P2, I agree with Aron that in general being more uncertain leads to a more diversified portfolio of interventions being optimal. I think while the point you raise—the future of complex systems like global geopolitics is hard to predict—is pretty obviously true, I resist the next step you take to say that using probabilities is useless/bad. Not to the same degree perhaps, but predicting whether a startup will succeed is very complex and difficult: it involves the personalities and dynamics of the founders and employees, the possibility of unforeseen technological breakthroughs, the performance of competitors including competitors that don’t yet exist, the wider economy and indeed geopolitical and trade relationships. And yet people make these predictions all the time, and some venture capitalists make more money than others and I think it is reasonable to believe we can make more and less informed guesses. Likewise with your questions, I think some probabilities we could use are more wrong than others. So I think like in any other domain, we make a best guess, think about how uncertain we are and use that to inform whether to spend more time thinking about it to narrow our uncertainty. My guess is you believe after making our initial probability guess based mainly on intuition it is hard to make progress and narrow our uncertainty. I think this could well be true. I suppose I just don’t see a better competing method of making decisions that avoids assigning probabilities to theory questions. My claim is that people/institutions that foreswear the use of probabilities will do systematically worse than those that try their best to predict the future, acknowledging the inherent difficulty of the endeavour.
Plausibly we can reformulate this premise as claiming that preventing the first nuclear deployment is more tractable because preventing escalation has more unknowns. I would be on board with this. I think it is mainly the anti-probability stance I am pushing back against.
For P3, I share the intuition that being cavalier about the deaths of millions of people seems bad. I think as a minimum it is bad optics, and quite possibly more (corrosive to our empathy, and maybe empathy is useful as altruists?). It seems trivially true that taking resources that would be used to try to prevent a first deployment and incinerating those resources is negative EV (ie “de-emphasizing the goal … has adverse effects”). It is a much stronger more controversial claim that moving resources from escalation prevention to first-use prevention is good, and I feel ill-equipped to assess this claim.
Also, as a general point, I think these three premises are not actually premises in a unified logical structure (ie ‘all forum posts are in English, this is a forum post, therefore it is in English’) but rather quite independent arguments pointing to the same conclusion. Which is fine, just misleading to call them each premises in the one argument I think. FInally, I am not that sure of your internal history but one worry would be if you decided long ago intuitively based on the cultural milieu that the right answer is ‘the best intervention in nuclear policy is to try to prevent first use’ and then subconsciously sought out supporting arguments. I am not saying this is what happened or that you are any more guilty of this than me or anyone else, just that it is something I and we all should be wary of.
“We are not able to sponsor US employment visas for participants” from https://www.openphilanthropy.org/open-philanthropy-technology-policy-fellowship/
Given this, I assume for people with no connection to the US (not citizens, no green card etc) there is no point in applying?
This seems like an important point to make in the main post as it rules out probably the majority of people opening this post.
I downvoted because there are lots of questions lumped in together without enough motivation and cohesion for my liking, and compared to e.g. the moral weights project the engagement with these subtle issues feels more flippant than serious.
Hmm yes policy work is tricky—probably even harder to model in a CEA than the more physical interventions I was mainly thinking about. I suppose this is what I was gesturing at with “So plausibly EAs that have significant sway over government decision-makers and can convince them to invest more in flood defences should do this. This would be the case for almost any good policy though: if it is low-effort to convince the government to do it, you should.” But yes perhaps I did undersell the value of policy here. I think I mostly stand by my claim that if you are able to influence policy a lot you should probably focus on other things first. If as you say the policies needed for flooding are unusually tractable then yes that would change things.
Woohoo, wonderful news and thanks for your efforts!
I am curious about the impact on allocating funding between worldviews. The substantial reduction in longtermist funding should raise the value of the marginal longtermist grant, and thus change the optimal allocation between longtermism, global health, and animals. But does the worldview-diversification type approach preclude this sort of reallocation as the funding situation in a cause area changes?