...it seems like your argument is saying “(A) and (B) are both really hard to estimate, and they’re both really low likelihood—but neither is negligible. Thus, we can’t really know whether our interventions are helping. (With the implicit conclusion being: thus, we should be more skeptical about attempts to improve the long-term future)”

Thanks, that is fairly accurate summary of one of the crucial points I am making except I would also add that the **difficulty of estimation increases with time**. And this is a major concern here because the case of longtermism **rests precisely on there being greater and greater number of humans** (and other sentient independent agents) as the horizon of time expands.

Sometimes we can’t know the probability distribution of (A) vs. (B), but sometimes we can do better-than-nothing estimates, and for some things (e.g., some aspects of X-risk reduction) it seems reasonable to try.

Fully agree that we should try but the case of longtermism remains rather weak until we have some estimates and bounds that can be reasonably justified.

I didn’t get the intuition behind the initial formulation:

EU(f)=αminp∈CEpU(f)+(1−α)maxp∈CEpU(f)

What exactly is that supposed to represent? And what was the basis for assigning numbers to the contingency matrix in the two example cases you’ve considered?