Sure, I agree that most people’s actions have a streak of self-interest, and that posterity could serve as this even in cases of sacrificing your life. I took OP to be making a stronger claim, that it is simply wrong to say that “people have altruistic values” as well.
There’s just something up with saying that these altruistic actions are caused by selfish/social incentives, where the strongest such incentive is ostracism or the death penalty for doing it.
How does this reduction account for the many historical examples of people who defied local social incentives, with little hope of gain and sometimes even destruction? (Off the top of my head: Ignaz Semmelweis, Irena Sendler, Sophie Scholl.)
We can always invent sufficiently strange posthoc preferences to “explain” any behaviour: but what do you gain in exchange for denying the seemingly simpler hypothesis “they had terminal values independent of their wellbeing”?
(Limiting this to atheists, since religious martyrs are explained well by incentives.)
Actually I think Feynman has the same risk. (Consider his motto: “disregard others” ! All very well, if you’re him.)
I think I would have benefitted from Hanson’s ‘Elephant in the Brain’, since I was intensely frustrated by (what I saw as) pervasive, inexplicable, wilfully bad choices, and this frustration affected my politics and ethics.
But it’s high-risk, since it’s easy to misread as justifying adolescent superiority (having ‘seen through’ society).
I suggest randomising in two blocks: people who strongly prefer video calls vs people who strongly prefer text, with abstainers assigned to either. Should prevent one obvious failure mode, people having an incompatible medium.
I was sure that Kurzweil would be one, but actually he’s still on track. (“Proper Turing test passed by 2029”).
I wonder if the dismissive received view on him is because he states specific years (to make himself falsifiable), which people interpret as crankish overconfidence.
Fair. But without tech there would be much less to fight for. So it’s multiplicative.
Good call. I’d add organised labour if I was doing a personal accounting.
We could probably have had trans rights without Burou’s surgeries and HRT but they surely had some impact, bringing it forward(?)
No, I don’t have a strong opinion either way. I suspect they’re ‘wickedly’ entangled. Just pushing back against the assumption that historical views, or policy views, can be assumed to be unempirical.
Is your claim (that soc > tech) retrospective only? I can think of plenty of speculated technologies that swamp all past social effects (e.g. super-longevity, brain emulation, suffering abolitionism) and perhaps all future social effects.
[80k and others claim that]: “The development and introduction of disruptive new technologies is a more fundamental and important driver of long-term change than socio-political reform or institutional change.”
1) Why do you assume this is ideological (in the sense of not being empirically grounded)?
Anecdotally, I was a protest-happy socialist as a teenager, but changed my mind after reading about the many historical failures, grasping the depth of the calculation problem, and so on. This at least felt like an ideological shift dependent on facts.
2) 80,000 Hours have been pushing congressional staffer and MP as top careers (literally #1 or #2 on the somewhat-deprecated quiz) for years. And improving institutions is on their front page of problem areas.
This comment is a wonderful crystallisation of the ‘defensive statistics’ of Andrew Gelman, James Heathers and other great epistemic policemen. Thanks!
Thanks for this. I’m not very familiar with the context, but let me see if I understand. (In a first for me, I’m not sure whether to ask you to cite more scripture or add more formal argument.) Let’s assume a Christian god, and call a rational consequence-counting believer an Optimising Christian.
Your overall point is that there are (or might be) two disjoint ethics, one for us and one for God, and that ours has a smaller scope, falling short of long-termism, for obvious reasons. Is this an orthodox view?
1. “The Bible says not to worry, since you can trust God to make things right. Planning is not worrying though. This puts a cap on the intensity of our longterm concern.”
2. “Humans are obviously not as good at longtermism as God, so we can leave it to Him.”
3. “Classical theism: at least parts of the future are fixed, and God promised us no (more) existential catastrophes. (Via flooding.)”
4. “Optimising Christians don’t need to bring (maximally many) people into existence: it’s supererogatory.” But large parts of Christianity take population increase very seriously as an obligation (based on e.g. Genesis 1:28 or Psalm 127). Do you know of doctrine that Christian universalism stops at present people?
5. “Optimising Christians only need to ‘satisfice’ their fellows, raising them out of subsistence. Positive consequentialism is for God.” This idea has a similar structure to negative utilitarianism, a moral system with an unusual number of philosophical difficulties. Why do bliss or happiness have no / insufficient moral weight? And, theologically: does orthodoxy say we don’t need to make others (very) happy?
If I understand you, in your points (1) through (4) you appeal to a notion of God’s agency outside of human action or natural laws. (So miracles only?) But a better theology of causation wouldn’t rely on miracles, instead viewing the whole causal history of the universe as constituting God’s agency. That interpretation, which at least doesn’t contradict physics, would keep optimising Christians on the hook for x-risk.
Many of your points are appropriately hedged—e.g. “it might also be God’s job”—but this makes it difficult to read off actions from the claims. (You also appeal to a qualitative kind of Bayesian belief updating, e.g. “significant but not conclusive reason”.) Are you familiar with the parliamentary model of ethics? It helps us act even while holding nuanced/confused views—e.g. for the causation question I raised above, each agent could place their own subjective probabilities on occasionalism, fatalism, hands-off theology and so on, and then work out what the decision should be. This kind of analysis could move your post from food-for-thought into a tool for moving through ancient debates and imponderables.
Yudkowsky once officiated at a wedding. I find it quite beautiful.
More engineering than science, but Turing’s 1944 ‘Delilah’ system for portable speech encipherment had no equivalent for more than a decade. (His biographer claims “30 years” but I don’t know what he’s comparing it to.) It was never deployed and was classified by the British government, so it had no impact.
People probably won’t give those examples here, for civility reasons. The SSC post linked above covers some practices Greg probably means, using historical examples.
One great example is the pain gap / access abyss. Only coined around 2017, got some attention at EA Global London 2017 (?), then OPIS stepped up. I don’t think the OPIS staff were doing a cause-neutral search for this (they were founded 2016) so much as it was independent convergence.
Related news for the suffering engineering idea (but sadly also related for the cognition engineering one).
I really like this, particularly how the conceptual splits lead to the appropriate mitigations.
The best taxonomy of uncertainty I’ve ever seen is this great paper by some physicists reflecting on the Great Recession. It’s ordinal and gives a bit more granularity to the stats (“Opaque”) branch of your tree, and also has a (half-serious) capstone category for catching events beyond reason:
1. “Complete certainty”. You are in a Newtonian clockwork universe with no residuals, no observer effects, utterly stable parameters. So, given perfect information, you yield perfect predictions.
2. “Risk without uncertainty”. You know a probability distribution for an exhaustive set of outcomes. No statistical inference needed. This is life in a hypothetical honest casino, where the rules are transparent and always followed. This situation bears little resemblance to financial markets.
3. “Fully Reducible Uncertainty”. There is one probability distribution over a set of known outcomes, but parameters are unknown. Like an honest casino, but one in which the odds are not posted and must therefore be inferred from experience. In broader terms, fully reducible uncertainty describes a world in which a single model generates all outcomes, and this model is parameterized by a finite number of unknown parameters that do not change over time and which can be estimated with an arbitrary degree of precision given enough data. As sample size increases, classical inference brings this down to level 2.
4. “Partially Reducible Uncertainty”. The distribution generating the data changes too frequently or is too complex to be estimated, or it consists in several nonperiodic regimes. Statistical inference cannot ever reduce this uncertainty to risk. Four sources:(1) stochastic or time-varying parameters that vary too frequently to be estimated accurately; (2) nonlinearities too complex to be captured by existing models, techniques, and datasets; (3) non-stationarities and non-ergodicities that render useless the Law of Large Numbers, Central Limit Theorem, and other methods of statistical inference and approximation; and (4) the dependence on relevant but unknown and unknowable conditioning information...
5. “Irreducible uncertainty”. Ignorance so complete that it cannot be reduced using data: no distribution, so no success in risk management. Such uncertainty is beyond the reach of probabilistic reasoning, statistical inference, and any meaningful quantification. This type of uncertainty is the domain of philosophers and religious leaders, who focus on not only the unknown, but the unknowable.
There’s a small, new literature analysing the subset of nonexistence I think you mean, under the name “impossible worlds”. (The authors have no moral or meta-ethical aims.) It might help to use their typology of impossible situations: Impossible Ways vs Logic Violators vs Classical Logic Violators vs Contradiction-Realizers.
To avoid confusion, consider ‘necessarily-nonexistent’ or ‘impossible moral patients’ or some new coinage like that, instead of just ‘nonexistent beings’ otherwise people will think you’re talking about the old Nonidentity Problem.
I think you’ll struggle to make progress, because the intuition that only possible people can be moral patients is so strong, stronger than the one about electrons or microbial life and so on. In the absence of positive reasons (rather than just speculative caution), the project can be expected to move attention away from moral patients to nonpatients—at least, your attention.
Meta: If you don’t want to edit out the thirteen paragraphs of preamble, maybe add a biggish summary paragraph at the top; the first time I read it (skimming, but still) I couldn’t find the proposition.
Ah I see. Agreed—thanks for clarifying.