What is ? It seems all the work is being done by having in the exponent.

# Michael_Wiebe

How about this: fanaticism is fine in principle, but in practice we never face any actual fanatical choices. For any actions with extremely large value V, we estimate p < 1/V, so that the expected value is <1, and we ignore these actions based on standard EV reasoning.

in order to assess the value (or normative status) of a particular action we can in the first instance just look at the long-run effects of that action (that is, those after 1000 years), and then look at the short-run effects just to decide among those actions whose long-run effects are among the very best.

Is this not laughable? How could anyone think that “looking at the 1000+ year effects of an action” is workable?

What are the comparative statics for how uncertainty affects decisionmaking? How does a decisionmaker’s behavior differ under some uncertainty compared to no uncertainty?

Consider a social planner problem where we make transfers to maximize total utility, given idiosyncratic shocks to endowments. There are two agents, , with endowments (with probability 1) and So either gets nothing or twice as much as .

We choose a transfer to solve:

For a baseline, consider and . Then we get an optimal transfer of . Intuitively, as , (if B gets 10 for sure, don’t make any transfer from A to B), and as (if B gets 0 for sure, split A’s endowment equally).

So that’s a scenario with risk (known probabilities), but not uncertainty (unknown probabilities). What if we’re uncertain about the value of ?

Suppose we think , for some distribution over . If we maximize expected utility, the problem becomes:

Since the objective function is linear in probabilities, we end up with the same problem as before, except with instead of . If we know the mean of , we plug it in and solve as before.

So it turns out that this form of uncertainty doesn’t change the problem very much.

Questions:

- if we don’t know the mean of , is the problem simply intractable? Should we resort to maxmin utility?

- what if we have a hyperprior over the mean of ? Do we just take another level of expectations, and end up with the same solution?

- how does a stochastic dominance decision theory work here?

Do you think Will’s three criteria are inconsistent with the informal definition I used in the OP (“what most matters about our actions is their very long term effects”)?

In my setup, I could say for some large ; ie, generations contribute basically nothing to total social utility . But I don’t think this captures longtermism, because this is consistent with the social planner allocating no resources to safety work (and all resources to consumption of the current generation); the condition puts no constraints on . In other words, this condition only matches the first of three criteria that Will lists:

(i) Those who live at future times matter just as much, morally, as those who live today;

(ii) Society currently privileges those who live today above those who will live in the future; and

(iii) We should take action to rectify that, and help ensure the long-run future goes well.

I’m a bit skeptical about the value of formal modelling here. The parameter estimates would be almost entirely determined by your assumptions, and I’d expect the confidence intervals to be massive.

I think a toy model would be helpful for framing the issue, but going beyond that (to structural estimation) seems not worth it.

and also a world where shorttermism is true

On Will’s definition, longtermism and shorttermism are mutually exclusive.

Suppose you’re taking a one-off action , and then you get (discounted) reward

I’m a bit confused by this setup. Do you mean that is analogous to , the allocation for ? If so, what are you assuming about ? In my setup, I can compare to. , so we’re comparing against the optimal allocation, holding fixed .

where is some large number.

I’m not sure this works. Consider: this condition would also be satisfied in a world with no x-risk, where each generation becomes successively richer and happier, and there’s no need for present generations to care about improving the future. (Or are you defining as the marginal utility of on generation , as opposed to the utility level of generation under ?)

My model here is riffing on Jones (2016); you might look there for solving the model.

Re infinite utility, Jones does say (fn 6): “As usual, must be sufficiently large given growth so that utility is finite.”

# Formalizing longtermism

**Assumption Based Planning**– having a written version of an organization’s plans and then identify load-bearing assumptions and assessing the vulnerability of the plan to each assumption.**Exploratory Modeling**– rather than trying to model all available data to predict the most likely outcome these models map out a wide range of assumptions and show how different assumptions lead to different consequences**Scenario planning**[2]

Can you clarify how these tools are distinct? My (ignorant) first impression is that they just boil down to “use critical thinking”.

Re algebra, are you defending the numbers you gave as reasonable? Otherwise, if we’re just making up numbers, might as well do the general case.

Would ‘countercyclical altruism’ also capture this view?

I think this would be easier to explain with a two-sector model: ie, just and . Also, would it be easier to just work with algebra? Ie, .

Assuming a budget of 6 units

How does this fit with ? That’s 10 units, no?

I will assume, for simplicity, constant marginal cost-effectiveness across each domain/effect/worldview

It’s worth emphasizing that this assumption rules out the diminishing returns case for diversifying; this is a feature, since we want to isolate the uncertainty-case for diversifying.

One version of the phase change model that I think is worth highlighting: S-curve growth.

Basically, the set of transformative innovations is finite, and we discovered most of them over the past 200 years. Hence, the Industrial Revolution was a period of fast technological growth, but that growth will end as we run out of innovations.The hockey-stick graph will level out and become an S-curve, as .

Although, is it the case that growth(GDP) increased during the modern era (ie, growth(population) has been rising)? My recollection is that the IR was a structural break, with jumping from 0.5% to 2% (or something).

Right, growth(GDP) > growth(GDP per capita) when growth(population)>0.

while the author agrees that growth rates have been increasing in the modern era (roughly, the Industrial Revolution and everything after)

I think this is a misunderstanding. The common view is that the growth rate has been constant in the modern era.

Yes, I’m saying that it happens to be the case that, in practice, fanatical tradeoffs never come up.

Hm, doesn’t claiming V=∞ also require perfect certainty? Ie, to know that V is literally infinite rather than some large number.