ben_chugg

• ben_chuggDec 27, 2020, 4:00 PM

  0 points

  0 ∶ 0

  in reply to: james’s comment on: Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

  Well, far be it from me to tell others how to spend their time, but I guess it depends on what the goal is. If the goal is to literally put a precise number (or range) on the probability of nuclear war before 2100, then yes, I think that’s a fruitless and impossible endeavour. History is not an iid sequence of events. If there is such a war, it will be the result of complex geopolitical factors based on human belief, desires, and knowledge at the time. We cannot pretend to know what these will be. Even if you were to gather all the available evidence we have on nuclear near misses, and generate some sort of probability based on this, the answer would look something like:

  “Assuming that in 2100 the world looks the same as it did during the time of past nuclear near misses, and nuclear misses are distributionally similar to actual nuclear strikes, and [a bunch of other assumptions], then the probability of a nuclear war before 2100 is x”.

  We can debate the merits of such a model, but I think it’s clear that it would be of limited use.

  None of this is to say that we shouldn’t be working on nuclear threat, of course. There are good arguments for why this is a big problem that have nothing to do with probability and subjective credences.

• ben_chuggDec 27, 2020, 3:39 PM

  1 point

  0 ∶ 0

  in reply to: Owen Cotton-Barratt’s comment on: Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

  Hi Owen!

  Re: inoculation of criticism. Agreed that it doesn’t make criticism impossible in every sense (otherwise my post wouldn’t exist). But if one reasons with numbers only (i.e., EV reasoning), then longtermism becomes unavoidable. As soon as one adopts what I’m calling “Bayesian epistemology”, then there’s very little room to argue with it. One can retort: Well, yes, but there’s very little room to argue with General Relativity, and that is a strength of the theory, not a weakness. But the difference is that GR is very precise: It’s hard to argue with because it aligns so well with observation. But there are lots of observations which would refute it (if light didn’t bend around stars, say). Longtermism is difficult to refute for a different reason, namely because it’s so easy to change the underlying assumptions. (I’m not trying to equate moral theories with empirical theories in every sense, but this example gets the point across I think.)

  Your second point does seem correct to me. I think I try to capture this sentiment when I say

  Greaves and MacAskill argue that we should have no moral discount factor, i.e., a “zero rate of pure time preference”. I agree — but this is besides the point. While time is morally irrelevant, it is relevant for solving problems.

  Here I’m granting that the moral view that future generations matter could be correct. But this, on my problem/​knowledge-focused view of progress, is irrelevant for decision making. What matters is maintaining the ability to solve problems and correct our (inevitable) errors.

• ben_chuggDec 27, 2020, 6:00 AM

  1 point

  0 ∶ 0

  in reply to: JackM’s comment on: Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

  Hi Jack,

  I think you’re right, the comparison to astrology isn’t entirely fair. But sometimes one has to stretch a little bit to make a point. And the point, I think, is important. Namely, that these estimates can be manipulated and changed all too easily to fit a narrative. Why not half a quadrillion, or 10 quadrillion people in the future?

  On the falsifiability point—I agree that the claims are technically falsifiable. I struggled with the language for this reason while writing it (and Max Heitmann helpfully tried to make this point before, but apparently I ignored him). In principle, all of their claims are falsifiable (if we go extinct, then sure, I guess we’ll know how big the future will be). Perhaps it’s better if I write “easily varied” or “amenable to drastic change” in place of irrefutable/​unfalsifiable?

  The great filter example is interesting, actually. For if we’re working in a Bayesian framework, then surely we’d assign such a hypothesis a probability. And then the number of future people could again be vast in expectation.

• ben_chuggDec 27, 2020, 5:35 AM

  3 points

  0 ∶ 0

  in reply to: ben_chugg’s comment on: Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

  As a major aside—there’s a little joke Vaden and I tell on the podcast sometimes when talking about Bayesianism vs Criticial Rationalism (an alternative philosophy first developed by Karl Popper). The joke is most certainly a strawman of Bayesianism, but I think it gets the point across.

  Bob and Alice are at the bar, being served by Carol. Bob is trying to estimate whether Carol has children. He starts with a prior of ¹⁄₂. He then looks up the base rate of adults with children, and updates on that. Then he updates based on Carol’s age. And what car she drives. And the fact that she’s married. And so on. He pulls out a napkin, does some complex math, and arrives at the following conclusion: It’s 64.745% likely that Carol has children. Bob is proud of his achievement and shows the napkin to Alice. Alice leans over the bar and asks “Hey Carol—do you have kids?”.

  Now, obviously this is not how the Bayesian acts in real life. But it demonstrates the care the Bayesian takes in having correct beliefs; about having the optimal brain state. I think this is the wrong target. Instead, we should be seeking to falsify as many conjectures as possible, regardless of where the conjectures came from. I don’t care what Alice thought the probability was before she asked the question, only about the result of the test.

• ben_chuggDec 27, 2020, 5:17 AM

  1 point

  0 ∶ 0

  in reply to: james’s comment on: Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

  Hey James!

  Answering this in its entirety would take a few more essays, but my short answer is: When there are no data available, I think subjective probability estimates are basically useless, and do not help in generating knowledge.

  I emphasize the condition when there are no data available because data is what allows us to discriminate between different models. And when data is available, well, estimates become less subjective.

  Now, I should say that I don’t really care what’s “reasonable” for someone to do—I definitely don’t want to dictate how someone should think about problems. (As an aside, this is a pet peeve of mine when it comes to Bayesianism -it tells you how you must think in order to be a rational person. As if rationality was some law of nature to be obeyed.) In fact, I want people thinking about problems in many different ways. I want Eliezer Yudkowski applying Bayes’ rule and updating in strict accordance with the rules of probability, you being inspired by your fourth grade teacher, and me ingesting four grams of shrooms with a blindfold on in order to generate as many ideas as possible. But how do we discriminate between these ideas? We subject them to ruthless criticism and see which ones stand up to scrutiny. Assigning numbers to them doesn’t tell us anything (again, when there’s no underlying data).

  In the piece I’m making a slightly different argument to the above, however. I’m criticizing the tendency for these subjective estimates to be compared with estimates derived from actual data. Whether or not someone agrees with me that Bayesianism is misguided, I would hope that they still recognize the problem in comparing numbers of the form “my best guess about x” with “here’s an average effect estimate with confidence intervals over 5 well-designed RTCs”.

Strong Longter­mism, Ir­refutabil­ity, and Mo­ral Progress

• ben_chuggDec 18, 2020, 5:32 PM

  5 points

  0 ∶ 0

  in reply to: EJT’s comment on: A case against strong longtermism

  Hi Elliott, just a few side comments from someone sympathetic to Vaden’s critique:

  I largely agree with your take on time preference. One thing I’d like to emphasize is that thought experiments used to justify a zero discount factor are typically conditional on knowing that future people will exist, and what the consequences will be. This is useful for sorting out our values, but less so when it comes to action, because we never have such guarantees. I think there’s often a move made where people say “in theory we should have a zero discount factor, so let’s focus on the future!”. But the conclusion ignores that in practice we never have such unconditional knowledge of the future.

  Re: the dice example:

  First, your point about future expectations being undefined seems to prove too much. There are infinitely many ways of rolling a fair die (someone shouts ‘1!’ while the die is in the air, someone shouts ‘2!’, etc.). But there is clearly some sense in which I ought to assign a probability of ¹⁄₆ to the hypothesis that the die lands on 1.

  True—there are infinitely many things that can happen while the die is in the air, but that’s not the outcome space about which we’re concerned. We’re concerned about the result of the roll, which is a finite space with six outcomes. So of course probabilities are defined in that case (and in the 6 vs 20 sided die case). Moreover, they’re defined by us, because we’ve chosen that a particular mathematical technique applies relatively well to the situation at hand. When reasoning about all possible futures however, we’re trying to shoehorn in some mathematics that is not appropriate to the problem (math is a tool—sometimes it’s useful, sometimes it’s not). We can’t even write out the outcome space in this scenario, let alone define a probability measure over it.

  So, to summarise the above, we have to assign probabilities to empirical hypotheses, on pain of getting Dutch-booked and accuracy-dominated. And all reasonable-seeming probability assignments imply that we should pursue longtermist interventions.

  Once you buy into the idea that you must quantify all your beliefs with numbers, then yes—you have to start assigning probabilities to all eventualities, and they must obey certain equations. But you can drop that framework completely. Numbers are not primary—again, they are just a tool. I know this community is deeply steeped in Bayesian epistemology, so this is going to be an uphill battle, but assigning credences to beliefs is not the way to generate knowledge. (I recently wrote about this briefly here.) Anyway, the Bayesianism debate is a much longer one (one that I think the community needs to have, however), so I won’t yell about any longer, but I do want to emphasize that it is only one way to reason about the world (and leads to many paradoxes and inconsistencies, as you all know).

  Appreciate your engagement :)

• ben_chuggDec 17, 2020, 8:39 PM

  18 points

  0 ∶ 0

  in reply to: Owen Cotton-Barratt’s comment on: A case against strong longtermism

  Hi Owen! Really appreciate you engaging with this post. (In the interest of full disclosure, I should say that I’m the Ben acknowledged in the piece, and I’m in no way unbiased. Also, unrelatedly, your story of switching from pure maths to EA-related areas has had a big influence over my current trajectory, so thank you for that :) )

  I’m confused about the claim

  I don’t think they’re saying (and I certainly don’t think) that we can ignore the effects of our actions over the next century; rather I think those effects matter much more for their instrumental value than intrinsic value.

  This seems in direct opposition to what the authors say (and what Vaden quoted above), namely that:

  The idea, then, is that for the purposes of evaluating actions, we can in the first instance often simply ignore all the effects contained in the first 100 (or even 1000) years

  I understand that they may not feel this way, but it is what they argued for and is, consequently, the idea that deserves to be criticized. Next, you write that if

  we had certainty in some finite time horizon (however large), then all of the EVs would become defined again and this technical objection would disappear.

  I don’t think so. The “immeasurability” of the future that Vaden has highlighted has nothing to do with the literal finiteness of the timeline of the universe. It has to do, rather, with the set of all possible futures (which is provably infinite). This set is immeasurable in the mathematical sense of lacking sufficient structure to be operated upon with a well-defined probability measure. Let me turn the question around on you: Suppose we knew that the time-horizon of the universe was finite, can you write out the sample space, $\sigma$-algebra, and measure which allows us to compute over possible futures?

  Finally, I’m not sure what to make of

  e.g. if someone tried the reasoning from the Shivani example in earnest rather than as a toy example in a philosophy paper I think it would rightly get a lot of criticism

  When reading their paper, I honestly did not read it as a toy example. And I don’t believe the authors state it as such. When discussing Shivani’s options they write:

  Our remaining task, then, is to show that there does indeed exist at least one option available to Shivani with the property that its far-future expected value (over BAU) is significantly greater than the best available short-term expected value (again relative to BAU). That is the task of the remainder of this section.

  and when discussing AI risk in particular:

  There is also a wide consensus among diverse leading thinkers (both within and outside the AI Research community) to the effect that the risks we have just hinted at are indeed very serious ones, and that much more should be done to mitigate them.

  Considering that the Open Philanthropy Project has poured millions into AI Safety, that it’s listed as a top cause by 80K, and that EA’s far-future-fund makes payouts to AI safety work, if Shivani’s reasoning isn’t to be taken seriously then now is probably a good time to make that abundantly clear. Apologies for the harshness in tone here, but for an august institute like GPI to make normative suggestions in its research and then expect no one to act on them is irresponsible.

  Anyway, I’m a huge fan of 95% of EA’s work, but really think it has gone down the wrong path with longtermism. Sorry for the sass—much love to all :)