There’s an accompanying column in The Guardian:
Running with MacAskill’s line of reasoning, we asked participants in this week’s Guardian Essential poll to think through whether future time horizons would be positive or negative for humanity (although we confined our frame to a relatively conservative ten millennia).
If I were debating you on the topic, it would be wrong to say that you think it’s a Pascal’s mugging. But I read your post as being a commentary on the broader public debate over AI risk research, trying to shift it away from “tiny probability of gigantic benefit” in the way that you (and others) have tried to shift perceptions of EA as a whole or the focus of 80k. And in that broader debate, Bostrom gets cited repeatedly as the respectable, mainstream academic who puts the subject on a solid intellectual footing.
(This is in contrast to MIRI, which as SIAI was utterly woeful and which in its current incarnation still didn’t look like a research institute worthy of the name when I last checked in during the great Tumblr debate of 2014; maybe they’re better now, I don’t know.)
In that context, you’ll have to keep politely telling people that you think the case is stronger than the position your most prominent academic supporter argues from, because the “Pascal’s mugging” thing isn’t going to disappear from the public debate.
The New Yorker writer got it straight out of this paper of Bostrom’s (paragraph starting “Even if we use the most conservative of these estimates”). I’ve seen a couple of people report that Bostrom made a similar argument at EA Global.
I get what you’re saying, but, e.g., in the recent profile of Nick Bostrom in the New Yorker:
No matter how improbable extinction may be, Bostrom argues, its consequences are near-infinitely bad; thus, even the tiniest step toward reducing the chance that it will happen is near-infinitely valuable. At times, he uses arithmetical sketches to illustrate this point. Imagining one of his utopian scenarios—trillions of digital minds thriving across the cosmos—he reasons that, if there is even a one-per-cent chance of this happening, the expected value of reducing an existential threat by a billionth of a billionth of one per cent would be worth a hundred billion times the value of a billion present-day lives. Put more simply: he believes that his work could dwarf the moral importance of anything else.
While the most prominent advocate in the respectable-academic part of that side of the debate is making Pascal-like arguments, there’s going to be some pushback about Pascal’s mugging.
I confess I’m a bit surprised no one else has linked this yet
Judging by GiveWell’s Twitter and Facebook feeds, the post is mis-dated—it only went live about 8 hours ago (at time of writing my comment), rather than 2 or 3 days ago.
I think this is referring to a common probability question, e.g., example 3 here.
Thanks Peter! I’ll make the top-level post later today.
How did you do that so quickly?
(I might have given the impression that I did this all during a weekend. This isn’t quite right—I spent 2-3 evenings, about 8 hours in total, going from the raw csv files to nice and compact .js function. Then I wrote the plotter on the weekend.)
I did this bit in Excel. If the money amounts were in column A, I insert three columns to the right: B for the currency (assumed USD unless otherwise specified), C for the min of the range given, D for the max. In column C, I started with =IF(ISNUMBER(A2), A2, “”) and dragged that formula down the column. Then I went through line by line, reading off any text entries, and turning them into currency/min/max (if a single value was reported, I entered it as the min, and left the max blank). currency, tab, number, enter, currency, tab, number, tab, number, enter, currency, tab...
It’s not a fun way to spend an evening (hence why I didn’t do the lifetime donations as well), but it doesn’t actually take that long.
Then: new column E for the AVERAGE(C2:D2) dragged down the column. Then I typed in the average currency conversions for 2013 into a new sheet and did a lookup (most users would use VLOOKUP I think, I used MATCH and OFFSET) to get my final USD numbers in column F.
Also, do you have the GitHub code for your plotter?
As a fierce partisan of the “_final_really_2” school of source control, I’m yet to learn how to GitHub. You can view the Javascript source easily enough though, and save it locally. (I suggest deleting the Google Analytics “i,s,o,g,r,a,m” script if you do this, or your browser might go looking for Google in your file system for a few seconds before plotting the graphs). The two scripts not in the HTML file itself are d3.min.js and ea_survey.data.js. (EDIT: can’t be bothered fixing the markdown underscores here.)
A zip file with my ready-to-run CSV file and the R script to turn it into a Javascript function is here.
I’ve made a bar chart plotter thing with the survey data: link.
The first 17 entries in imdata.csv have some mixed-up columns, starting (at latest) from
Have you volunteered or worked for any of the following organisations? [Machine Intelligence Research Institute]
until (at least)
Over 2013, which charities did you donate to? [Against Malaria Foundation].
Some of this I can work out (volunteering at “6-10 friends” should obviously be in the friends column), but the blank cells under the AMF donations have me puzzled.
Thanks for this, and thanks for putting the full data on github. I’ll have a sift through it tonight and see how far I get towards processing it all (perhaps I’ll decide it’s too messy and I’ll just be grateful for the results in the report!).
I have one specific comment so far: on page 12 of the PDF you have rationality as the third-highest-ranking cause. This was surprisingly high to me. The table in imdata.csv has it as “Improving rationality or science”, which is grouping together two very different things. (I am strongly in favour of improving science, such as with open data, a culture of sharing lab secrets and code, etc.; I’m pretty indifferent to CFAR-style rationality.)
A hero means roughly what you’d expect—someone who takes personal responsibility for solving world problems. Kind of like an effective altruist.
What I understand about rationality ‘heroes’ is limited to what I’ve gleaned from Miranda’s post, but to me it seems like earning to give fits much more naturally into a sidekick category than into a hero category.
Why doesn’t it bother you at all that a theory has counterintuitive implications in counterfactual scenarios? Shouldn’t this lower your confidence in the theory?
I think my disagreement is mostly on (1) -- I expect that a correct moral theory would be horrendously complicated. I certainly can’t reduce my moral theory to some simple set of principles: there are many realistic circumstances where my principles clash (individual rights versus greater good, say, or plenty of legal battles where it’s not clear what a moral decision would be), and I don’t know of any simple rules to decide what principles I deem more important in which situations. Certainly there are many realistic problems which I think could go either way.
But I agree that all other things equal, simplicity is a good feature to have, and enough simplicity might sometimes outweigh intuition. Perhaps, once future-me carefully consider enormous aggregative ethics problems, I will have an insight that allows a drastically simplified moral theory. The new theory would solve the repugnant conclusion (whatever I think ‘repugnant’ means in this future!). Applied to present-me’s day-to-day problems, such a simplified theory will likely give slightly different answers to what I think today: maybe the uncertainty I have today about certain court cases would be solved by one of the principles that future-me thinks of.
But I don’t think the answers will change a lot. I think my current moral theory basically gives appropriate answers (sometimes uncertain ones) to my problems today. There’s wiggle-room in places, but there are also some really solid intuitions that I don’t expect future-me to sacrifice. Rescuing the drowning child (at least when I live in a world without the ability to create large numbers of sentient beings!) would be one of these.
Maybe I’ve misinterpreted ‘repugnant’ here? I thought it basically meant “bad”, but Google tells me that a second definition is “in conflict or incompatible with”, and now that I know this, I’m guessing that it’s the latter definition that you are using for ‘repugnant’. But I’m finding it difficult to make sense of it all (it carries a really strong negative connotation for me, and I’m not sure if it’s supposed to in this context—there might be nuances that I’m missing), so I’ll try to describe my position using other words.
If my moral theory, when applied to some highly unrealistic thought experiment (which doesn’t have some clear analog to something more realistic), results in a conclusion that I really don’t like, then:
I accept that my moral theory is not a complete and correct theory; and
this is not something that bothers me at all. If the thought experiment ever becomes relevant, I’ll worry about how to patch up the theory then. In the meantime, I’ll carry on trying to live by my moral theory.
If you find this implication repugnant, {you should also find it repugnant that a theory would have that implication if you found yourself in that position, even if as a matter of fact you don’t}.
I reject the implication inside the curly brackets that I added. I don’t care what would happen to my moral theory if creating these large populations becomes possible; in the unlikely event that I’m still around when it becomes relevant, I’m happy to leave it to future-me to patch up my moral theory in a way that future-me deems appropriate.
As an analogy
I guess I could attach some sort of plausibility score to moral thought experiments. Rescuing a drowning child gets a score near 1, since rescue situations really do happen and it’s just a matter of detail about how much it costs the rescuer. As applied to donating to charity, the score might have to be lowered a little to account for how donating to charity isn’t an exact match for the child in the pond.
The Nazi officials case… seems pretty plausible to me? Like didn’t that actually happen?
Something of a more intermediate case between the drowning child and creating large populations would be the idea of murdering someone to harvest their organs. This is feasible today, but irrelevant since no-one is altruistically murdering people for organs. I think it’s reasonable for someone previously a pure utilitarian to respond with, “Alright, my earlier utilitarianism fails in this case, but it works in lots of other places, so I’ll continue to use it elsewhere, without claiming that it’s a complete moral theory.” (And if they want to analyse it really closely and work out the boundaries of when killing one person to save others is moral and when not, then that’s also a reasonable response.)
A thought experiment involving the creation of large populations gets a plausibility score near zero.
Can you (or anyone else who feels similarly) clarify the sense in which you consider the repugnant conclusion ‘not actually important’, but the drowning child example ‘important’?
Because children die of preventable diseases, but no-one creates arbitrarily large populations of people with just-better-than-nothing well-being.
OK—I mean the hybrid theory—but I see two possibilities (I don’t think it’s worth my time reading up on this subject enough to make sure what I mean matches exactly the terminology of the paper(s) you refer to):
In my hybridisation, I’ve already sacrificed some intuitive principles (improving total welfare versus respecting individual rights, say), by weighing up competing intuitions.
Whatever counter-intuitive implications my mish-mash, sometimes fuzzily defined hybrid theory has, they have been pushed into the realm of “what philosophers can write papers on”, rather than what is actually important. The repugnant conclusion falls under this category.
Whichever way it works out, I stick resolutely to saving the drowning child.
If that procedure was followed consistently, it would disprove all moral theories.
I consider this a reason to not strictly adhere to any single moral theory.
Hopefully this is my last comment in this thread, since I don’t think there’s much more I have to say after this.
I don’t really mind if people are working on these problems, but it’s a looooong way from effective altruism.
Taking into account life-forms outside our observable universe for our moral theories is just absurd. Modelling our actions as affecting an infinite number of our descendants feels a lot more reasonable to me. (I don’t know if it’s useful to do this, but it doesn’t seem obviously stupid.)
Many-worlds is even further away from effective altruism. (And quantum probabilities sum to 1 anyway, so there’s a natural way to weight all the branches if you want to start shooting people if and only if a photon travels through a particular slit and interacts with a detector, ….)
Letting the child drown in the hope that
a) there’s an infinite number of life-forms outside our observable universe, and
b) that the correct moral theory does not simply require counting utilities (or whatever) in some local region
strikes me as far more problematic. More generally, letting the child drown is a reductio of whatever moral system led to that conclusion.
Speaking from my geographically distant perspective: I definitely saw it as a leader-led shift rather than coming from the rank-and-file. There was always a minority of rank-and-file coming from Less Wrong who saw AI risk as supremely important, but my impression was that this position was disproportionately common in the (then) Centre for Effective Altruism, and there was occasional chatter on Facebook (circa 2014?) that some people there saw the global poverty cause as a way to funnel people towards AI risk.
I think the AI-risk faction started to assert itself more strongly in EA from about 2015, successfully persuading other major leader figures one by one over the following years (e.g. Holden in 2016, as linked to by Carl). But by then I wasn’t following EA closely, and I don’t have a good sense of the timeline.