Thanks for this, important work! I read the forum post but not the longer report or the accompanying code, so apologies if you have addressed these there. A few thoughts/questions:
Without doing or even reading any of the maths, lots of the results seemed quite intuitive/unsurprising to me. Eg that X-risk and shrimp interventions will look worse if we are ambiguity-averse or really want to avoid not making any difference. Do you also find the results fairly unsurprising; what is the most surprising part? I’m not necessarily arguing that unsurprising results would mean the formal maths wasn’t worth doing, but I think it is at least relevant.
I was confused by the “periods of counterfactual impact” for X-risk interventions. Would this just be the length of time civilisation continues after extinction is averted? If so, capping this at 10^5 years seems quite low.
I don’t think you talked about the near-term vs long-term X-risk intervention in the forum post before the table of results.
I think I either misunderstood or disagreed with this putative contrast: “are our actions valuable only if they prevent an existential or non-existential catastrophe that would have otherwise occurred, or is it valuable to merely lower the probability of a catastrophe?” The first option seems to me to be a non-starter. Surely we can never know whether our action prevented an existential catastrophe. Maybe the paper I write inspires someone to work on some problem who then goes on to make a breakthrough without which we would have all died, but if we don’t all die we won’t know this. It also seems strange to me to need to wait until things have panned out to be able to assess the moral value of an action. I want to know now what to do, not wait a few years to see what happened in one possible world and whether an existential catastrophe was prevented. In summary, I think we should use our ex-ante probabilities rather than waiting to see what happens ex-post.
This is probably beyond your scope, but I am wondering what the psychological basis for these various types of risk aversion are. One idea I had is that for many of us donations are at least partially intended to boost our social status, and so depending on what your social group finds admirable, the risk-averse place to donate might be a religious institution, a local homeless shelter, a political party, your alma mater, Greenpeace, AMF, etc. So on this model people are implicitly trying to maximise the number of relevant people who will find their choice of place to donate praiseworthy. I think this could partially explain ambiguity aversion (if the chance of Shrimp welfare is quite ambiguous, some people will think the probability is far lower than I do, and think I am a fool for donating there) and difference-making (if this road safety advocacy org falls in a heap and achieves nothing, people will think I was silly to donate to it in the first place). I think insofar as we are impartial altruists with regards to our donations, this psychological explanation being ~true should make us less sympathetic to risk-aversion. Avoiding the worst doesn’t really fit into this story of mine.
I think this sort of data could be quite nicely and helpfully displayed through a simple web app where users can tweak the intervention, risk-model, and key parameters. I didn’t realise how easy these were to make, until my supervisor told me about the shiny package for R (something similar may exist for python, or you can easily port the data over to R), and it turned out ChatGPT could convert my underlying code into a functional UI very well. Mine is here, on an unrelated topic, in case you want inspiration: https://oscar-delaney.shinyapps.io/AMR-evolution/ and the code for it is at https://github.com/Oscar-Delaney/bacteriostatic
Thanks for this, important work! I read the forum post but not the longer report or the accompanying code, so apologies if you have addressed these there. A few thoughts/questions:
Without doing or even reading any of the maths, lots of the results seemed quite intuitive/unsurprising to me. Eg that X-risk and shrimp interventions will look worse if we are ambiguity-averse or really want to avoid not making any difference. Do you also find the results fairly unsurprising; what is the most surprising part? I’m not necessarily arguing that unsurprising results would mean the formal maths wasn’t worth doing, but I think it is at least relevant.
I was confused by the “periods of counterfactual impact” for X-risk interventions. Would this just be the length of time civilisation continues after extinction is averted? If so, capping this at 10^5 years seems quite low.
I don’t think you talked about the near-term vs long-term X-risk intervention in the forum post before the table of results.
I think I either misunderstood or disagreed with this putative contrast: “are our actions valuable only if they prevent an existential or non-existential catastrophe that would have otherwise occurred, or is it valuable to merely lower the probability of a catastrophe?” The first option seems to me to be a non-starter. Surely we can never know whether our action prevented an existential catastrophe. Maybe the paper I write inspires someone to work on some problem who then goes on to make a breakthrough without which we would have all died, but if we don’t all die we won’t know this. It also seems strange to me to need to wait until things have panned out to be able to assess the moral value of an action. I want to know now what to do, not wait a few years to see what happened in one possible world and whether an existential catastrophe was prevented. In summary, I think we should use our ex-ante probabilities rather than waiting to see what happens ex-post.
This is probably beyond your scope, but I am wondering what the psychological basis for these various types of risk aversion are. One idea I had is that for many of us donations are at least partially intended to boost our social status, and so depending on what your social group finds admirable, the risk-averse place to donate might be a religious institution, a local homeless shelter, a political party, your alma mater, Greenpeace, AMF, etc. So on this model people are implicitly trying to maximise the number of relevant people who will find their choice of place to donate praiseworthy. I think this could partially explain ambiguity aversion (if the chance of Shrimp welfare is quite ambiguous, some people will think the probability is far lower than I do, and think I am a fool for donating there) and difference-making (if this road safety advocacy org falls in a heap and achieves nothing, people will think I was silly to donate to it in the first place). I think insofar as we are impartial altruists with regards to our donations, this psychological explanation being ~true should make us less sympathetic to risk-aversion. Avoiding the worst doesn’t really fit into this story of mine.
I think this sort of data could be quite nicely and helpfully displayed through a simple web app where users can tweak the intervention, risk-model, and key parameters. I didn’t realise how easy these were to make, until my supervisor told me about the shiny package for R (something similar may exist for python, or you can easily port the data over to R), and it turned out ChatGPT could convert my underlying code into a functional UI very well. Mine is here, on an unrelated topic, in case you want inspiration: https://oscar-delaney.shinyapps.io/AMR-evolution/ and the code for it is at https://github.com/Oscar-Delaney/bacteriostatic