tylermjohn

Karma: 740

tylermjohn Mar 17, 2025, 3:33 PM
3 points
0 ∶ 1
in reply to: William_MacAskill’s comment on: Existential Choices Symposium with Will MacAskill and other special guests (3-5pm GMT Monday)
The easiest way, in my view, to make a near-optimal future very likely, conditional on non-extinction, is if value is bounded above.
Yes.
Of all the issues facing classical utilitarianism, this seems the most problematic to me.
So you doubt fanaticism, the view that a tiny chance of an astronomically good outcome can be more valuable than a certainty of a decent outcome. What about in the case of certainty? Do you doubt the utilitarian’s objection to Common-sense Eutopia? This kind of aggregation seems important for the case for longtermism. (See the pages of paper dolls in What We Owe the Future.)

tylermjohn Mar 17, 2025, 3:29 PM
3 points
0 ∶ 0
in reply to: Greg_Colbourn ⏸️ ’s comment on: Existential Choices Symposium with Will MacAskill and other special guests (3-5pm GMT Monday)
Suppose p(doom) is 90%. Then preventing extinction multiplies the value of the world by 10 in expectation.
But suppose that the best attainable futures are 1000 times better than the default non-extinction scenario. Then ensuring we are on track to get the best possible future multiplies the value of the world by 100 in expectation, even after factoring in the 90% chance of extinction.
In this toy model, you should only allocate your resources to reducing extinction if it is 10 times more tractable than ensuring we are on track to get the best possible future, at the current margin.
You might think that we can just defer this to the future. But I’ve assumed in the set-up that the default future is 1/1000th as good as the best future. So apparently our descendants are not going to be very good at optimising the future, and we can’t trust them with this decision.
Where do you think this goes wrong?

tylermjohn Mar 17, 2025, 3:24 PM
5 points
0 ∶ 0
in reply to: OscarD🔸’s comment on: Existential Choices Symposium with Will MacAskill and other special guests (3-5pm GMT Monday)
There are funky axiologies where value is superlinear in resources — basically any moral worldview that embraces holism. If you think that the whole, arranged a particular way, is more valuable than the parts, then you will be even more precious about how precisely the world should be arranged than the total utilitarian.

tylermjohn Mar 17, 2025, 3:15 PM
7 points
0 ∶ 0
on: Existential Choices Symposium with Will MacAskill and other special guests (3-5pm GMT Monday)
Position statement: I chose 36% disagreement. AMA!
My view is that Earth-originating civilisation, if we become spacefaring, will attain around 0.0001% of all value. This still makes extinction risk astronomically valuable (it’s equivalent to optimising a millionth of the whole cosmos!), but if we could increase the chance of optimising 1% of the universe by 1%, this would be 100x more valuable than avoiding extinction. (You’re not going to get an extremely well grounded explanation of these numbers from me, but I hope they make my position clearer.)
My view is that if Earth-originating civilisation becomes spacefaring, over the long term we will settle the entire universe and use nearly all of its energy. However, we could use this energy for many different things. I expect that by default we will mostly use this energy to create copies of ourselves (human brains) or digital slaves that allow us to learn and achieve new things, since humans love nothing more than humans and Earthly things. But these are inefficient media for realising moral value. We could instead use this energy to realise vastly more value, as I argue in Power Laws of Value. But our descendants basically won’t care at all about doing this. So I expect that we will miss out on nearly all value, despite using most of the universe’s energy.
I think that there will be significant path dependency in what Earth-originating civilisation chooses to make with our cosmic endowment. In particular, I expect that artificial intelligence will drive most growth and most choices after the 21st century. Several factors, such as the number of distinct agents, their values, and the processes by which their values evolve over time will therefore make a decisive difference to what our descendants choose to do with the cosmos.
So our choices about AI governance and architecture today are likely to make a significant and, at least on some paths we can choose, predictable difference to what our descendants do with all of the energy in the universe. If we do this well, it could make the difference between attaining almost no value and nearly all value.
Given that I expect us to miss out on almost all value by default, I view the value of avoiding extinction as smaller than those who think we will achieve almost all value by default.

tylermjohn Mar 17, 2025, 2:37 PM
3 points
0 ∶ 0
in reply to: zdgroff’s comment on: Power Laws of Value
Thanks, @zdgroff! I think MPL is most important if you think that there are going to be some agents shaping things, these agents’ motivations are decisive for what outcomes are achieved, and you might (today) be able to align these agents with tail-valuable outcomes. Then aligning these agents with your moral values is wildly important. And by contrast marginal improvements to the agents’ motivations are relatively unimportant.
You’re right that if you don’t have any chance of optimizing any part of the universe, then MPL doesn’t matter as much. Do you think that there won’t be agents (even groups of them) with decisive control over what outcomes are achieved in (even parts of) the world?
It seems to me in the worst case we could at least ask Dustin to try to buy one star and then eventually turn it into computronium.

tylermjohn Mar 17, 2025, 1:51 PM
3 points
0 ∶ 0
in reply to: Gideon Futerman’s comment on: Power Laws of Value
Thanks a lot, @Gideon Futerman! Good additions, which all seem right to me.
Another question I’ve had on my mind is how much MPL is related to additive separability. You might at a first pass think that moral atomism makes you more likely to buy MPL, since you have so many different spaces of value to optimise. But holistic views can in principle lead to even sharper differences in the value of worlds — for example, you could have a view that says that you need to align all of the stars forever or you don’t capture any value.
I’d like to have a clearer view on when moral viewpoints will tend to end up at MPL, but I don’t have one yet.

tylermjohn Mar 17, 2025, 1:26 PM
3 points
0 ∶ 0
in reply to: Gideon Futerman’s comment on: Power Laws of Value
Nice point. I shouldn’t have picked averageism as the most extreme version of this view. It would have been more apt to pick a “capped” model where the value on additional utility (or utility of a specific type) becomes zero after enough of it has been achieved.

tylermjohn Mar 17, 2025, 12:23 PM
3 points
0 ∶ 0
in reply to: Davidmanheim’s comment on: Power Laws of Value
Thanks a lot @Davidmanheim!
definitely resonates with the empirical EA view of power-law returns, which I was surprised you didn’t mention.
Thanks, I meant to mention this but I think it got cut when I revised the second section for clarity, alas.
1. The version of non-naturalist moral realism that the divergence seems both very strong, and strange to me. It assumes that the true moral code is unlike mathematical realism, where it’s accessible with reflection and would be a natural conclusion for those who cared.
Thanks for this. I’d love to hear non-naturalist moral realists talk about how they think moral facts are epistemically accessible, if it’s not just luck. (Some philosophers do explicitly assume it’s luck.) I think the problem here is extremely hard, including for mathematics, and my own view on mathematics is closest to Millian empiricism (we learn basic math e.g. arithmetic by observing physics and for more advanced mathematics we freely choose axioms that we test against reality for usefulness and explanatory power).
The best philosopher writing on the epistemology of both mathematics and ethics is Justin Clarke-Doane, who combines a form of pluralist realism about mathematics with expressivism about ethics.
However we access moral facts, I’d expect my points about dependency on initial conditions to generalise.
2. “You could accept diminishing returns to value in utility… but you’re unlikely to be a longtermist, laser focused on extinction risk if you do.” I think this is false under the view of near-term extinction risk that is held by most of those who seem concerned about AI extinction risk, or even varieties of the hinge-of-history view whereby we are affected in the near term by longtermist concerns.
True, you could accept this moral view and also accept that:
- X-risk is so high and so tractable that reducing it is better than GiveWell on relatively NT grounds
- Animals don’t matter much, so don’t beat NT x-risk
And then you’d avoid the objection.
Or you could think utility doesn’t matter much after you’ve got, say, 10^12 humans, and then x-risk still looks good but making sure the right future happens looks less important.
In general I think I was too quick here, good catch.

tylermjohn Mar 17, 2025, 10:01 AM
1 point
0 ∶ 0
on: Discussion Thread: Existential Choices Debate Week
I’m compressing two major dimensions into one here:
- Expected value (I think EV of maxevas is vastly higher than EV of maxipok)
- Robustness (the case for maxevas is highly empirically theory laden and values dependent)

tylermjohn Nov 12, 2023, 7:12 PM
2 points
1 ∶ 0
on: EA Explorer GPT: A New Tool to Explore Effective Altruism
I haven’t tried this, but I’m excited about the idea! Effective Altruism as an idea seems unusually difficult to communicate faithfully, and creating a GPT that can be probed on various details and correct misconceptions seems like a great way to increase communication fidelity.

tylermjohn Sep 29, 2023, 5:17 PM
6 points
0 ∶ 0
on: Net global welfare may be negative and declining
On your future directions / tentative reflections (with apologies that I haven’t looked into your model, which is probably cool and valuable!):

To the extent that we think this is relevant for things like lock-in and x-risk prioritisation we need to also think that current trends are predictive of future trends. But it’s not at all clear that they are once you take into account the possibility of explosive growth a la https://www.cold-takes.com/all-possible-views-about-humanitys-future-are-wild/. Moreover, worlds where there is explosive growth have way more moral patients, so if their probability is non-negligible they tend to dominate moral considerations.

Once we focus on explosive growth scenarios as the most important, I find much more persuasive considerations like these: https://www.effectivealtruism.org/articles/the-expected-value-of-extinction-risk-reduction-is-positive

I’ve written up decently extended reflections on why we shouldn’t give much weight to the fact that the history and presency of our world is an utter moral hellscape that I’m happy to share privately if these questions are important for you.

(All that said, I do think lock-in is undervalued in longtermism and I’m excited to see more work on that, and I do think the path to x-risk prioritisation is much more complicated than many EAs think and that these kinds of considerations you point out are exactly why.)

tylermjohn Sep 22, 2023, 3:34 PM
2 points
0 ∶ 0
in reply to: spencerg’s comment on: Should Effective Altruists be Valuists instead of utilitarians?
I’ve only skimmed the essay but it looks pretty good! Many of the ideas I had in mind are covered here, and I respond very differently to this than to your post here.

I don’t know what most EAs believe about ethics and metaethics, but I took this post to be about the truth, or desirability of these metaethical, ethical, and methodological positions, not whether they’re better than what most EAs believe. And that’s what I’m commenting on here.

tylermjohn Sep 22, 2023, 11:00 AM
17 points
5 ∶ 1
on: Should Effective Altruists be Valuists instead of utilitarians?
Hi Spencer and Amber,

There’s a pretty chunky literature on some of these issues in metaethics, e.g.:
- Moral fictionalism, or why it could make sense to talk in terms of moral truths even if there aren’t any
- Moral antirealism/constructivism, or why there can be moral “shoulds” and “oughts” even if these are just mental attitudes
- Why even if you’re a pluralist, utilitarian considerations dominate your reasoning on a range of psychologically typical value systems given how much welfare matters to people compared to other things, and how much we can affect it given effective altruism
- How there can be different ways of valuing things, some that you endorse and some that you don’t (especially among constructivists like Street, Korsgaard, and Velleman), and why it could make sense to only act on values you endorse acting on
- Relatedly, how your moral theory might be different from your spontaneous sentiments because you can think through these sentiments and bring them into harmony (e.g. the discussion of reflective equilibrium)
Obviously it would be a high bar to have to have a PhD level training on these topics and read through the whole literature before posting on the EA Forum, so I’m not suggesting that! But I think it would be useful to talk some of these ideas through with antirealist metaethicists because they have responses to a bunch of these criticisms. I know Spencer and I chatted about this once and probably we should chat again! I could also refer you to some other EA folks who would be good to chat to about this, probably over DM.

All of that said, I do think there are useful things about what you’re doing here, especially e.g. part 2, and I do think that some antirealist utilitarianism is mistaken for broadly the reasons you say! And the philosophers definitely haven’t gotten everything right, I actually think most metaethicists are confused. But especially claims like those made in part 3 have a lot of good responses in existing discussion.

ETA: Actually if you’re still in NYC one person I’ll nominate to chat with about this topic is Jeff Sebo.

tylermjohn Jul 30, 2023, 2:44 PM
8 points
2 ∶ 0
on: Why isn’t there a charity evaluator for longtermist projects?
I’d be pretty excited to see a new platform for retail donors giving to x-risk charities. For this, you’d want to have some x-risk opportunities that are highly scaleable (can do ≥ $10m p.a., will execute the project over years reliably without intervention or outside pressure), measurable (you can write out a legible, robust, well-quantifed theory of change from marginal dollars to x-risk), have a pretty smooth returns curve (so people can have decent confidence that their donations have the returns that they expect, whether they are a retail donors or a large donor). And then you could build out cost effectiveness models given different assumptions about values (e.g. time preference, population ethics) and extinction risk models that people might have, and offer retail donors a few different ways of thinking about and modeling the impacts of their donations (e.g. lives saved, micro- or picodooms).

I’d guess there are some bio interventions that fit these criteria. For AI safety there could be a crowd-funded compute cluster for safety research. This could be pretty boring and difficult to model robustly unless there was good reporting on the wins that came out of research using the cluster and further iteration on the model based on the track record.

tylermjohn May 18, 2023, 9:53 PM
11 points
4 ∶ 0
on: U.S. Regulatory Updates to Benefit-Cost Analysis: Highlights and Encouragement to Submit Public Comments
Noting that I think that making substantive public comments on this draft (including positive comments about what it gets right) is one of the very best volunteer opportunities for EAs right now! I plan to send a comment on the draft before the deadline of 6 June.

tylermjohn Apr 1, 2023, 9:05 AM
9 points
1 ∶ 0
in reply to: Sam Anschell’s comment on: What’s surprised me as an entry-level generalist at Open Phil & my recommendations to early career professionals
Thanks Sam! I don’t have much more to say about this right now since on a couple things we just have different impressions, but I did talk to someone at 80k last night about this. They basically said: some people need the advice Tyler gave, some people need the advice Sam gave. The best general advice is probably “apply broadly”: apply to some EA jobs, to some high-impact jobs outside of EA, to some upskilling jobs, etc. And then pick the highest EV job you were accepted to (where EV is comprehensive and includes things like improvements to your future career from credentialing and upskilling).

tylermjohn Mar 31, 2023, 3:02 AM
73 points
20 ∶ 2
on: What’s surprised me as an entry-level generalist at Open Phil & my recommendations to early career professionals
Hi readers! I work as a Programme Officer at a longtermist organisation. (These views are my own and don’t represent my employer!) I think there’s some valuable advice in this post, especially about not being constrained too much by what you majored in. But after running several hiring rounds, I would frame my advice a bit differently. Working at a grantmaking organisation did change my views on the value of my time. But I also learned a bunch of other things, like:
1. The majority of people who apply for EA jobs are not qualified for them.
2. Junior EA talent is oversupplied, because of management constraints, top of funnel growth, and because EAs really want to work at EA organisations.
3. The value that you bring to your organisation/to the world is directly proportional to your skills and your fit for the role.
Because of this, typically when I talk to junior EAs my advice is not to apply to lots more EA jobs but rather to find ways of skilling up — especially by working at a non-EA organisation that has excellent managers and invests in training its staff — so that one can build key skills that make one indispensable to EA organisations.

Here’s a probably overly strong way of stating my view that might bring the point home: try to never apply to EA jobs, and instead get so good at something that EA orgs will headhunt you and fight over you.

I know that there are lots of nice things about working at EA organisations (culture, community, tangible feelings of impact) but if you really value work at EA organisations, then you should value highly skilled work at EA organisations even more (I think a lot more!). Having more junior EAs find ways to train up their skills and spend less time looking for EA work is the only way I can see to convert top of funnel community growth into healthy middle of funnel community growth.

tylermjohn Mar 16, 2023, 12:40 PM
4 points
0 ∶ 0
on: Write a Book?
I’m not sure if this fits your concept, but it might be helpful to have a guidebook that caters specifically to new EAs, to help give guidance to people excited about the ideas but unsure how to put them into practice in daily life, in order to convert top of funnel growth into healthy middle of funnel growth. This could maybe couple with a more general audience book that appeals to people who are antecedently interested in the ideas.

A couple things I’d like to see in this are the reasoning transparency stuff, guidance on going out and getting skills outside of the EA community to bring into the community, anti-burnout stuff, and various cultural elements that will help community health and epistemics.

tylermjohn Feb 25, 2023, 11:51 PM
2 points
3 ∶ 0
on: Updates from the Mental Health Funder’s Circle
Congrats, Will!

tylermjohn Jan 31, 2023, 12:56 PM
4 points
1 ∶ 0
on: Follow-Up Survey: Major Layoffs at Tech Giants [recruiting support]
I’ve just shared the survey. I think it would be useful if the survey included more information on who will use it, who the data will be available, who is running the survey, and the like.