vadmas

Karma: 173

A case against strong longtermism

vadmas15 Dec 2020 20:56 UTC

68 points

79 comments1 min readEA link

vadmas 16 Dec 2020 22:01 UTC
13 points
0 ∶ 0
in reply to: Owen Cotton-Barratt’s comment on: A case against strong longtermism
Hey Owen—thanks for your feedback! Just to respond to a few points -
>Your argument against expected value is a direct rebuttal of the argument for, but in my eyes this is one of your weaker criticisms.

Would be able to elaborate a bit on where the weaknesses are? I see in the thread you agree the argument is correct (and from googling your name I see you have a pure math background! Glad it passes your sniff-test :) ). If we agree EVs are undefined over possible futures, then in the Shivani example, this is like comparing 3 lives to NaN. Does this not refute at least 1 / 2 of the assumptions longtermism needs to ‘get off the ground’?

> Overall I feel like a lot of your critique is not engaging directly with the case for strong longtermism; rather you’re pointing out apparently unpalatable implications.

Just to comment here—yup I intentionally didn’t address the philosophical arguments in favor of longtermism, just because I felt that criticizing the incorrect use of expected values was a “deeper” critique and one which I hadn’t seen made on the forum before. What would the argument for strong longtermism look like without the expected value calculus? It’s my impression that EVs are central to the claim that we can and should concern ourselves with the future 1 billion years from now.

Also my hope was that this would highlight a methodological error (equating made up numbers to real data) that could be rectified, whether or not you buy my other arguments about longtermism. I’d be a lot more sympathetic with longtermism in general if the proponents were careful to adhere to the methodological rule of only ever comparing subjective probabilities with other subjective probabilities (and not subjective probabilities with objective ones, derived from data).

> I would welcome more work on understanding the limits of this kind of reasoning, but I’m wary of throwing the baby out with the bathwater if we say we must throw our hands up rather than reason at all about things affecting the future.

Yup totally—if you permit me a shameless self plug, I wrote about an alternative way to reason here.

> As a minor point, I don’t think that discounting the future really saves you from undefined expectations, as you’re implying.

Oops sorry no wasn’t implying that—two orthogonal arguments.

>I do think that if all people across time were united in working for the good

People are united across time working for the good! Each generation does what it can to make the world a little bit better for its descendants, and in this way we are all united.

vadmas 17 Dec 2020 5:04 UTC
3 points
0 ∶ 0
in reply to: Owen Cotton-Barratt’s comment on: A case against strong longtermism

Overall though I think that longtermism is going to end up with practical advice which looks quite a lot like “it is the duty of each generation to do what it can to make the world a little bit better for its descendants.”

Goodness, I really hope so. As it stands, Greaves and MacAskill are telling people that they can “simply ignore all the effects [of their actions] contained in the first 100 (or even 1000) years”, which seems rather far from the practical advice both you and I hope they arrive at.

Anyway, I appreciate all your thoughtful feedback—it seems like we agree much more than we disagree, so I’m going to leave it here :)

vadmas 18 Dec 2020 3:25 UTC
3 points
0 ∶ 0
in reply to: djbinder’s comment on: A case against strong longtermism
if we helped ourselves to some cast-iron guarantees about the size and future lifespan of the universe (and made some assumptions about quantization) then we’d know that the set of possible futures was smaller than a particular finite number (since there would only be a finite number of time steps and a finite number of ways of arranging all particles at each time step). Then even if I can’t write it down, in principle someone could write it down, and the mathematical worries about undefined expectations go away.
It certainly not obvious that the universe is infinite in the sense that you suggest. Certainly nothing is “provably infinite” with our current knowledge. Furthermore, although we may not be certain about the properties of our own universe, we can easily imagine worlds rich enough to contain moral agents yet which remain completely finite. For instance, you could image a cellular automata with a finite grid size and which only lasted for a finite duration.
Aarrrgggggg was trying to resist weighing in again … but I think there’s some misunderstanding of my argument here. I wrote:
The set of all possible futures is infinite, regardless of whether we consider the life of the universe to be infinite. Why is this? Add to any finite set of possible futures a future where someone spontaneously shouts “1”!, and a future where someone spontaneously shouts “2”!, and a future where someone spontaneously shouts “3!” (italics added)
A few comments:
- We’re talking about possible universes, not actual ones, so cast-iron guarantees about the size and future lifespan of the universe are irrelevant (and impossible anyway).
- I intentionally framed it as someone shouting a natural number in order to circumvent any counterargument based on physical limits of the universe. If someone can think it, they can shout it.
- The set of possible futures is provably infinite because the “shouting a natural number” argument established a one-to-one correspondence between the set of possible (triple emphasis on the word * possible * ) futures, and the set of natural numbers, which are provably infinite (see proof here ).
- I’m not using fancy or exotic mathematics here, as Owen can verify. Putting sets in one-to-one correspondence with the natural numbers is the standard way one proves a set is countably infinite. (See https://en.wikipedia.org/wiki/Countable_set).
- Physical limitations regarding the largest number that can be physically instantiated are irrelevant to answering the question “is this set finite or infinite”? Mathematicians do not say the set of natural numbers are finite because there are a finite number of particles in the universe. We’re approaching numerology territory here...
Okay this will hopefully be my last comment, because I’m really not trying to be a troll in the forum or anything. But please represent my argument accurately!

vadmas 18 Dec 2020 20:21 UTC
7 points
0 ∶ 0
in reply to: Isaac Dunn’s comment on: A case against strong longtermism
Hey Issac,
On this specific question, I have either misunderstood your argument or think it might be mistaken. I think your argument is “even if we assume that the life of the universe is finite, there are still infinitely many possible futures—for example, the infinite different possible universes where someone shouts a different natural number”.
But I think this is mistaken, because the universe will end before you finish shouting most natural numbers. In fact, there would only be finitely many natural numbers you could finish shouting before the universe ends, so this doesn’t show there are infinitely many possible universes.
Yup you’ve misunderstood the argument. When we talk about the set of all future possibilities, we don’t line up all the possible futures and iterate through them sequentially. For example, if we say it’s possible tomorrow might either rain, snow, or hail, we * aren’t * saying that it will first rain, then snow, then hail. Only one of them will actually happen.
Rather we are discussing the set of possibilities { $r a i n$ , $s n o w$ , $h a i l$ }, which has no intrinsic order, and in this case has a cardinality of 3.
Similarly with the set of all possible futures. If we let $f_{i}$ represent a possible future where someone shouts the number $i$ , then the set of all possible futures is { $f_{1}$ , $f_{2}$ , $f_{3}$ , … }, which has cardinality $\infty$ and again no intrinsic ordering. We aren’t saying here that a single person will shout all numbers between 1 and $\infty$ , because as with the weather example, we’re talking about what might possibly happen, not what actually happens.
More generally, I think I agree with Owen’s point that if we make the (strong) assumption the universe is finite in duration and finite in possible states, and can quantise time, then it follows that there are only finite possible universes, so we can in principle compute expected value.
No this is wrong. We don’t consider physical constraints when constructing the set of future possibilities—physical constraints come into the picture later. So in the weather example, we could include into our set of future possibilities something absurd, and which violates known laws of physics. For example we are free to construct a set like { $r a i n$ , $s n o w$ , $h a i l$ , $r a i n s_f r o g s$ }.
Then we factor in physical constraints by assigning probability 0 to the absurd scenario. For example our probabilities might be { $0.2, 0.4, 0.4, 0$ }.
But no laws of physics are being violated with the scenario “someone shouts the natural number i”. This is why this establishes a one-to-one correspondence between the set of future possibilities and the natural numbers, and why we can say the set of future possibilities is (at least) countably infinite. (You could establish that the set of future possibilities is uncountably infinite as well by having someone shout a single digit in Cantor’s diagonal argument, but that’s beyond what is necessary to show that EVs are undefined.
For example, I’d love to hear when (if at all) you think we should use expected value reasoning, and how we should make decisions when we shouldn’t.
Yes I think that the EV style-reasoning popular on this forum should be dropped entirely because it leads to absurd conclusions, and basically forces people to think along a single dimension.
So for example I’ll produce some ridiculous future scenario (Vaden’s x-risk: In the year 254 012 412 there will be a war over blueberries in the Qualon region of delta quadrant , which causes an unfathomable amount of infinite suffering ) and then say: great, you’re free to set your credence about this scenario as high or as low as you like.
But now I’ve trapped you! Because I’ve forced you to think about the scenario only in terms of a single 1 dimensional credence-slider. Your only move is to set your credence-slider really really small, and I’ll set my suffering-slider really really high, and then using EVs, get you to dedicate your income and the rest of your life to Blueberry-Safety research.
Note also that EV style reasoning is only really popular in this community. No other community of researchers reasons in this way, and they’re able to make decisions just fine. How would any other community reason about my scenario? They would reject it as absurd and be done with it. Not think along a single axis (low credence/high credence).
That’s the informal answer, anyway. Realizing that other communities don’t reason in this way and are able to make decisions just fine should at least be a clue that dropping EV style arguments isn’t going to result in decision-paralysis.
The more formal answer is to consider using an entirely different epistemology, which doesn’t deal with EVs at all. This is what my vague comments about the ‘framework’ were eluding to in the piece. Specifically, I have in mind Karl Popper’s critical rationalism, which is at the foundation of modern science. CR is about much more than that, however. I discuss what a CR approach to decision making would look like in this piece if you want some longer thoughts on it.
But anyway, I digress… I don’t expect people to jettison their entire worldview just because some random dude on the internet tells them to. But for anyone reading who might be curious to know where I’m getting a lot of these ideas from (few are original to me), I’d recommend Conjectures and Refutations. If you want to know what an alternative to EV style reasoning looks like, the answers are in that book.
(Note: This is a book many people haven’t read because think they already know the gist. “Oh, C&R! That’s the book about falsification, right?” It’s about much much more than that :) )

vadmas 18 Dec 2020 21:01 UTC
1 point
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
Yup, the latter. This is why the lack-of-data problem is the other core part of my argument. Once data is in the picture, now we can start to get traction. There is something to fit the measure to, something to be wrong about, and a means of adjudicating between which choice of measure is better than which other choice. Without data, all this probability talk is just idol speculation painted with a quantitative veneer.
What links here?
- Max_Daniel's comment on A case against strong longtermism by vadmas (18 Dec 2020 19:20 UTC; 41 points)
- Max_Daniel's comment on A case against strong longtermism by vadmas (19 Dec 2020 13:18 UTC; 5 points)

vadmas 18 Dec 2020 23:00 UTC
3 points
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
Yes, there are certain rare cases where longterm prediction is possible. Usually these involve astronomical systems, which are unique because they are cyclical in nature and unusually unperturbed by the outside environment. Human society doesn’t share any of these properties unfortunately, and long term historical prediction runs into the impossibility proof in epistemology anyway.

vadmas 18 Dec 2020 23:03 UTC
1 point
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
~~See discussion below w/ Flodorner on this point :)~~
You are Flodorner!

vadmas 19 Dec 2020 19:09 UTC
12 points
0 ∶ 0
on: A case against strong longtermism
Hi all! Really great to see all the engagement with the post! I’m going to write a follow up piece responding to many of the objections raised in this thread. I’ll post it in the forum in a few weeks once it’s complete—please reply to this comment if you have any other questions and I’ll do my best to address all of them in the next piece :)
What links here?
- Proving too much: A response to the EA forum by vadmas (15 Feb 2021 19:00 UTC; 24 points)

vadmas 21 Dec 2020 6:00 UTC
1 point
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
I don’t think I buy the impossibility proof as predicting future knowledge in a probabilistic manner is possible (most simply, I can predict that if I flip a coin now, that there’s a ⁵⁰⁄₅₀ chance I’ll know the coin landed on heads/tails in a minute).
In this example you aren’t predicting future knowledge, you’re predicting that you’ll have knowledge in the future—that is, in one minute, you will know the outcome of the coin flip. I too think we’ll gain knowledge in the future, but that’s very different from predicting the content of that future knowledge today. It’s the difference between saying “sometime in the future we will have a theory that unifies quantum mechanics and general relativity” and describing the details of future theory itself.
I am almost certain you won’t be able to find any rigorous mathematical proof for this intuition
The proof is here: https://vmasrani.github.io/assets/pdf/poverty_historicism_quote.pdf .
(And who said proofs have to be mathematical? Proofs have to be logical—that is, concerned with deducing true conclusions from true premises—not mathematical, although they often take mathematical form.)

vadmas 21 Dec 2020 22:07 UTC
2 points
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
Yep it’s Chapter 22 of The Open Universe (don’t have a pdf copy unfortunately)

vadmas 22 Dec 2020 5:30 UTC
2 points
0 ∶ 0
in reply to: Patrick’s comment on: A case against strong longtermism
Oops good catch, updated the post with a link to your comment.

vadmas 22 Dec 2020 16:52 UTC
1 point
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
Impressive write up! Fun historical note—in a footnote Popper says he got the idea of formulating the proof using prediction machines from personal communication with the “late Dr A. M. Turing”.

vadmas 22 Dec 2020 17:26 UTC
4 points
0 ∶ 0
in reply to: axioman’s comment on: A case against strong longtermism
I don’t see how we could predict anything in the future at all (like the sun’s existence or the coin flips that were discussed in other comments). Where is the qualitative difference between short- and long-term predictions?
Haha just gonna keep pointing you to places where Popper writes about this stuff b/c it’s far more comprehensive than anything I could write here :)

This question (and the questions re. climate change Max asked in another thread) are the focus of Popper’s book The Poverty of Historicism, where “historicism” here means “any philosophy that tries to make long-term predictions about human society” (i.e marxism, fascism, malthusianism, etc). I’ve attached a screenshot for proof-of-relevance:

(Ben and I discuss historicism here fwiw.) I have a pdf of this one, dm me if you want a copy :)

vadmas 22 Dec 2020 17:46 UTC
3 points
0 ∶ 0
in reply to: Max_Daniel’s comment on: A case against strong longtermism
Yes! Exactly! Hence why I keep bringing him up :)

vadmas 28 Dec 2020 8:57 UTC
2 points
0 ∶ 0
in reply to: Mau’s comment on: Strong Longtermism, Irrefutability, and Moral Progress
Hey Mauricio! Two brief comments -
Some others are focused on making decisions. From this angle, EV maximization and Bayesian epistemology were never supposed to be frameworks for creating knowledge—they’re frameworks for turning knowledge into decisions, and your arguments don’t seem to be enough for refuting them as such.
Yes agreed, but these two things become intertwined when a philosophy makes people decide to stop creating knowledge. In this case, it’s longtermism preventing the creation of moral and scientific knowledge by grinding the process of error correction to a halt, where “error correction” in this context means continuously reevaluating philanthropic organizations based on their near and medium term consequences, in order to compare results obtained against results expected.
Vaden offers a promising approach to making decisions, but it just passes the buck on this—we’ll still need an answer to my question when we get to his step 2
Both approaches pass on the buck, that’s why I defined ‘creativity’ here to mean: ‘whatever unknown software the brain is running to get out of the infinite regress problem.’ And one doesn’t necessarily need to answer your question, because there’s no requirement that the criticism take EV form (although it can).

vadmas 28 Dec 2020 20:08 UTC
1 point
0 ∶ 0
in reply to: Mau’s comment on: Strong Longtermism, Irrefutability, and Moral Progress
I don’t see how that gets you out of facing the question
Check out chapter 13 in Beginning of Infinity when you can—everything I was saying in that post is much better explained there :)

vadmas 28 Dec 2020 20:15 UTC
1 point
0 ∶ 0
in reply to: JackM’s comment on: Strong Longtermism, Irrefutability, and Moral Progress
Firstly, you and vadmas seem to assume number 2 is the case.
Oops nope the exact opposite! Couldn’t possibly agree more strongly with
Working on current problems allows us to create moral and scientific knowledge that will help us make the long-run future go well
Perfect, love it, spot on. I’d be 100% on board with longtermism if this is what it’s about—hopefully conversations like these can move it there. (Ben makes this point near the end of our podcast conversation fwiw)
Do you in fact think that knowledge creation has strong intrinsic value? I, and I suspect most EAs, only think knowledge creation is instrumentally valuable.
Well, both. I do think it’s intrinsically valuable to learn about reality, and I support research into fundamental physics, biology, history, mathematics, ethics etc for that reason. I think it would be intellectually impoverishing to only support research that has immediate and foreseeable practical benefits. But fortunately knowledge creation also has enormous instrumental value. So it’s not a one-or-the other thing.

vadmas 28 Dec 2020 20:27 UTC
1 point
0 ∶ 0
in reply to: AGB’s comment on: Strong Longtermism, Irrefutability, and Moral Progress
Yikes… now I’m even more worried … :|

vadmas 28 Dec 2020 20:38 UTC
6 points
0 ∶ 0
in reply to: weeatquince’s comment on: Strong Longtermism, Irrefutability, and Moral Progress
(as far as I can tell their entire point is that you can always do an expected value calculation and “ignore all the effects contained in the first 100” years)
Yes, exactly. One can always find some expected value calculation that allows one to ignore present-day suffering. And worse, one can keep doing this between now and eternity, to ignore all suffering forever. We can describe this using the language of “falsifiability” or “irrefutability” or whatever—the word choice doesn’t really matter here. What matters is that this is a very dangerous game to be playing.