Toby_Ord

Karma: 3,310

Toby_Ord Jul 11, 2024, 10:18 AM
8 points
0 ∶ 0
in reply to: Vasco Grilo🔸’s comment on: Shaping Humanity’s Longterm Trajectory
To be clear, I don’t think that most things are longtermist interventions with these permanent impacts (and most things aren’t trying to be).
Clearly some thing have had lasting effects. e.g. there were no electrical household items before the 19th century, and now they are ubiquitous and quite possibly will be with us as long as humanity lasts. Whereas if humanity had never electrified, the present would be very different.
That said, it is also useful to ask about the counterfactuals. e.g. if Maxwell or Edison or any other pioneer hadn’t made their discoveries, how different would we expect 2024 to be? In this case, it is less clear as someone else probably would have made these discoveries later. But then discoveries that build on them would have been delayed etc. I doubt that the counterfactual impact of Maxwell goes to zero until such a point as practically everything has been discovered (or everything downstream of electromagnetism). That said, it could easily have diminishing impact over time, as is typical of advancements (their impact does not scale with humanity’s duration).
But note that the paper actually doesn’t claim that a particular invention or discovery acts as an advancement. It suggests it is a possible model of the longterm impacts of things like that. One reason I bring it up is that it shows that even if it was a permanent advancement, then under a wide set of circumstances, it would still be beaten by reducing existential risk. And even if attempts to advance progress really were lasting advancements, the value could be negative if they also bring forward the end time.
I would love it if there was more investigation of the empirical measurement of such lasting effects, though it is outside of my field(s).

Toby_Ord Jul 1, 2024, 11:06 AM
17 points
3 ∶ 0
on: Detecting Genetically Engineered Viruses With Metagenomic Sequencing
Thanks for this excellent writeup of a very promising project.
A somewhat useful comparator would be to the proportion of people in Wuhan who were infected at the time that SARS-CoV 2 was detected. According to this report, the pathogen was identified on Jan 8, and by Jan 18, their estimate is that 2,500 – 6,100 of the 11 million people were infected (they don’t seem to report a number for Jan 8, though you might be able to work it out from their doubling time results). That is about 0.02% – 0.06%. So it sounds like you would need to scale this up by a factor of 20 or so to detect it as quickly as Wuhan detected their non-stealth corona virus.
Of course, SARS-CoV 2 wouldn’t have been found anywhere near as fast if it were a stealth pathogen, so this comparison only goes so far, but it is a start.

Toby_Ord Jun 6, 2024, 8:55 AM
3 points
1 ∶ 0
in reply to: Benjamin_Todd’s comment on: EA Dedicates
I really like your framing of “making doing good your central purpose”.
I also like the way the OP shows how we can embrace and support people of different levels of dedication, but I wonder whether having binary names (especially identity group names) is a good idea. Ultimately this is a spectrum (without the clear delineation and identity marker that comes in religions with monks) and it is probably good to have terms that reflect the continuum nature of it.

Toby_Ord May 16, 2024, 9:50 AM
4 points
0 ∶ 1
in reply to: Arepo’s comment on: Updates on the EA catastrophic risk landscape
I discuss something similar to that a bit on page 41, but mainly focusing on whether depletion could make it harder for civilisation to re-emerge. Ultimately, it still looks to me like it would be easier and faster the second time around.

Toby_Ord May 15, 2024, 3:50 PM
2 points
0 ∶ 0
in reply to: NunoSempere’s comment on: Robust longterm comparisons
I agree!
I didn’t want to get too distracted with these complication in the piece, but I’m sympathetic to these and other approaches to avoid the technical issue of divergent integrals of value when studying longterm effects.
In the case in question (where u(t) always equals k v(t)) we get an even stronger constraint the ratio of progressively longer integrals doesn’t just limit to a constant, but equals a constant.
There are some issues that come up with these approaches though. One is that they are all tacitly assuming that comparing things at a time is the right comparison. But suppose (contra my assumptions in the post) that population was always half as high in one outcome as the other. Then it may be doing worse at any time, but still have all the same people eventually come into existence and be equally good for all of them. Issues like this where the ratio depends on what variable is being integrated over don’t come up in the convergent integral cases.
All that said, the integrating to infinity in economic modelling is presumably not to be taken literally, and for any finite time horizon — no matter how mindbendingly large — my result that the discounting function doesn’t matter holds (even if the infinite integral were to diverge).

Toby_Ord May 15, 2024, 2:09 PM
10 points
0 ∶ 0
in reply to: Arepo’s comment on: Updates on the EA catastrophic risk landscape
For what it’s worth, my working assumption for many risks (e.g. nuclear, supervolcanic eruption) was that their contribution to existential risk via ‘direct’ extinction was of a similar level to their contribution via civilisation collapse. e.g. that a civilisation collapse event was something like 10 times as likely, but that there was also a 90% chance of recovery. So in total, the consideration of non-direct pathways roughly doubled my estimates for a number of risks.
One thing I didn’t do was to include their roles as risk factors. e.g. the effect that being on the brink of nuclear war has on overall existential risk even if the nuclear war doesn’t occur.
What links here?
- Vasco Grilo🔸's comment on Nuclear weapons safety and security—Career review by Benjamin Hilton (May 16, 2024, 5:16 PM; 0 points)

Toby_Ord Mar 4, 2024, 2:52 PM
8 points
0 ∶ 0
on: U.S. Regulatory Updates to Benefit-Cost Analysis: Highlights and Encouragement to Submit Public Comments
Thanks Danny!
For those who are unaware, Benefit Cost Analysis (BCA) is the main form of quantitative evaluation of cost-effectiveness in the US, UK, and beyond. Two of the biggest problems with it as a method are its counting of the value of a dollar equally for all people (which leads to valuing people themselves in proportion to their income) and a high, constant, discount rate. So the action on both of these are big improvements to quantitative priority setting in the US!

Toby_Ord Feb 27, 2024, 9:09 AM
4 points
1 ∶ 0
in reply to: Eevee🔹’s comment on: The Case for Animal-Inclusive Longtermism
Yes, I completely agree. When I was exploring questions about wild animal welfare almost 20 years ago, I was very surprised to see how the idea of thinking about individual animals’ lives was so foreign to the field.

Toby_Ord Feb 24, 2024, 11:50 AM
18 points
2 ∶ 0
on: The Case for Animal-Inclusive Longtermism
While I only had time to quickly read this piece, I agree with much of what I read and think it is a great contribution to the literature.
To clarify my own view, I think animals matter a great deal — now and over the longterm future. The focus on humanity in my work is primarily because we are the only moral agents we know of. In philosophical terms, this means that humanity has immense instrumental value. If we die, then as far as we know, there is nothing at all striving to shape the future of Earth (or our whole galaxy) towards what is good or just. It is with humanity in this role as moral agent, rather than moral patient, that I think the case for avoiding existential risk to humanity is at its most powerful. I don’t know whether the lion’s share of intrinsic value we create over the longterm future will be in the form of human flourishing, animal flourishing, or something else, and welcome much more discussion on that, with your paper being a good example.
As well as avoiding existential risk, I think that work to avoid locking in bad values or practices could also be very important on longtermist grounds, and that values connected to animals are good candidates.
I focused on what could happen to animal species rather than individual animals in those particular passages, but much of my thinking on animal ethics is in terms of individuals.
Like most people, I’m not sold on the idea that wild animal suffering makes the biosphere have negative overall value, such that ecosystem destruction would be good instead of bad (and so forth). But nor am I claiming that we should introduce animals to other planets. My point in those passages was to sketch the magnitude of the kinds of things humanity could achieve in terms of the environment and animal life. What to do with that power raises very big and very uncertain questions. My main claims were that we should protect our potential, and then think very long and hard about how best to fulfil it.

Toby_Ord Jan 5, 2024, 12:25 PM
41 points
16 ∶ 1
on: EA Wins 2023
Thanks Shakeel! This is an excellent post. There are so many big wins in the EA community that it can be hard to see the big picture and keep them all in mind. We all strive to see the big picture, but sometimes even for us, the latest drama can quickly drive the big successes from our memory. So big picture summaries like this are very useful.
Indeed, summaries of the whole period of EA achievements, or summaries that include setbacks as well as wins would also be good.

Toby_Ord Dec 29, 2023, 12:28 PM
17 points
4 ∶ 0
on: Zach Robinson will be CEA’s next CEO
Very exciting news! Welcome Zach — this is making me feel optimistic about EA in 2024.

Toby_Ord Nov 8, 2023, 3:58 PM
123 points
35 ∶ 0
on: 10 years of Earning to Give
Thanks so much for writing this, and even more for all you’ve done to help those less fortunate than yourself.
I’m glad I did that Daily Politics spot! It was very hard to tell in the early days how impactful media work was (and it still is!) so examples like this are very interesting.

Toby_Ord Sep 26, 2023, 11:39 AM
24 points
3 ∶ 0
in reply to: William_MacAskill’s comment on: William_MacAskill’s Shortform
Thanks so much for all your hard work on CEA/EV over the many years. You have been such a driving force over the years in developing the ideas, the community, and the institutions we needed to help make it all work well. Much of that work over the years has happened through CEA/EV, and before that through Giving What We Can and 80,000 Hours before we’d set up CEA to house them, so this is definitely in some sense the end of an era for you (and for EV). But a lot of your intellectual work and vision has always transcended the particular organisations and I’m really looking forward to much more of that to come!

Toby_Ord Sep 13, 2023, 5:02 PM
20 points
4 ∶ 0
in reply to: RyanCarey’s comment on: Thoughts on EA, post-FTX
Oh, I definitely agree that the guilt narrative has some truth to it too, and that the final position must be some mix of the two, with somewhere between a ¹⁰⁄₉₀ and ⁹⁰⁄₁₀ split. But I’d definitely been neglecting the ‘we got used’ narrative, and had assumed others were too (though aprilsun’s comment suggests I might be incorrect about that).
I’d add that for different questions related to the future of EA, the different narratives change their mix. For example, the ‘we got used’ narrative is at its most relevant if asking about ‘all EAs except Sam’. But if asking about whether it is good to grow EA, it is relevant that we may get more Sams. And if asking ‘how much good or bad do people who associate with EA do?’ the ‘guilt’ narrative increases in importance.

Toby_Ord Sep 13, 2023, 10:58 AM
24 points
7 ∶ 4
in reply to: Michael_PJ’s comment on: Thoughts on EA, post-FTX
This is a very interesting take, and very well expressed. You could well be right that the narrative that ‘we got used’ is the most correct simple summary for EAs/EA. And I definitely agree that it is an under-rated narrative. There could even be psychological reasons for that (EAs being more prone to guilt than to embarassment?).
I note that even if P(FTX exists | EA exists) were quite a bit higher than P(FTX exists | ~EA exists), that could be compatible with your suggested narrative of EAs being primarily marks/victims. To reuse your example, if you were the only person the perpetrator of the heist could con into lending their car to act as a getaway vehicle, then that would make P(Heist happens | Your actions) quite a bit higher than P(Heist happens | You acting differently), but you would still be primarily a mark or (minor) victim of the crime, rather than primarily one of the responsible parties for it.

Toby_Ord Sep 6, 2023, 9:09 PM
134 points
18 ∶ 2
on: Nick Beckstead is leaving the Effective Ventures boards
Nick is being so characteristically modest in his descriptions of his role here. He was involved in EA right from the start — one of the members of Giving What We Can at launch in 2009 — and he soon started running our first international chapter at Rutgers, before becoming our director of research. He contributed greatly to the early theory of effective altruism and along with Will and I was one of the three founding trustees of the Centre for Effective Altruism. I had the great pleasure of working with him in person for a while at Oxford University, before he moved back to the States to join Open Philanthropy. He was always thoughtful, modest, and kind. I’m excited to see what he does next.

Toby_Ord Aug 7, 2023, 9:54 AM
28 points
3 ∶ 0
on: General support for “General EA”
Thanks so much for writing this Arthur.
I’m still interested in the possibilities of changing various aspects of how the EA community works, but this post does a great job of explaining important things that we’re getting right already and — perhaps more importantly — in helping us remember what it is all about and why we’re here.

Toby_Ord Aug 2, 2023, 6:00 PM
2 points
0 ∶ 0
in reply to: William_MacAskill’s comment on: Shaping Humanity’s Longterm Trajectory
The term ‘humanity’ is definitely intended to be interpreted broadly. I was more explicit about this in The Precipice and forgot to reiterate it in this paper. I certainly want to include any worthy successors to homo sapiens. But it may be important to understand the boundary of what counts. A background assumption is that the entities are both moral agents and moral patients — capable of steering the future towards what matters and for being intrinsically part of what matters. I’m not sure if those assumptions are actually needed, but they were guiding my thought.
I definitely don’t intend to include alien civilisations or future independent earth-originating intelligent life. The point is to capture the causal downstream consequences of things in our sphere of control. So the effects of us on alien civilisations should be counted and any effects we have of on whether any earth species evolves after us, but it isn’t meant to be a graph of all value in the universe. My methods wouldn’t work for that, as we can’t plausibly speed that up, or protect it all etc (unless we were almost all the value anyway).

Toby_Ord Aug 2, 2023, 4:04 PM
4 points
0 ∶ 0
in reply to: William_MacAskill’s comment on: Shaping Humanity’s Longterm Trajectory
I think I may have been a bit too unclear about which things I found more promising than others. Ultimately the chapter is more about the framework, with a few considerations added for and against each of the kinds of idealised changes, and no real attempt to be complete about those or make all-things-considered judgments about how to rate them. Of the marginal interventions I discuss, I am most excited about existential-risk reduction, followed by enhancements.
As to your example, I feel that I might count the point where the world became permanently controlled by preference utilitarians an existential catastrophe — locking in an incorrect moral system forever. In general, lock-in is a good answer for why things might not happen later if they don’t happen now, but too much lock-in of too big a consequence is what I call an existential catastrophe. So your example is good as a non-*extinction* case, but to find a non-existential one, you may need to look for examples that are smaller in size, or perhaps only partly locked-in?

Toby_Ord Aug 2, 2023, 2:53 PM
7 points
0 ∶ 0
in reply to: William_MacAskill’s comment on: Shaping Humanity’s Longterm Trajectory
I think “open to speed-ups” is about right. As I said in the quoted text, my conclusion was that contingent speed-ups “may be possible”. They are not an avenue for long-term change that I’m especially excited about. The main reason for including them here was to distinguish them from advancements (these two things are often run together) and because they fall out very natural as one of the kinds of natural marginal change to the trajectory whose value doesn’t depend on the details of the curve.
That said, it sounds like I think they are a bit more likely to be possible than you do. Here are some comments on that.
One thing is that it is easier to have a speed-up relative to another trajectory than to have one that is contingent — which wouldn’t have happened otherwise. Contingent speed-ups are the ones of most interest to longtermists, but those that are overdetermined to happen are still relevant to studying the value of the future and where it comes from. e.g. if the industrial revolution was going to happen anyway, then the counterfactual value of it happening in the UK in the late 1700s may be small, but it is still an extremely interesting event in terms of dramatically changing the rate of progress from then onwards compared to a world without an industrial revolution.
The more natural thought is that, at some point in time, we either hit a plateau, or hit some hard limit of how fast v(.) can grow (perhaps driven by cubic or quadratic growth as future people settle the stars).
Even if v(.) hits a plateau, you can still have a speed-up, it is just that it only has an impact on the value achieved before we would have hit the value anyway. That could be a large change (e.g. if the plateau isn’t reached in the first 1% of our lifetime), but even if it isn’t, that doesn’t stop this being a speed-up, it is just that changing the speed of some things isn’t very valuable, which is a result that is revealed by the framework.
Suppose v(.) stops growing exponentially with progress and most of its increase is then governed by growing cubically as a humanity’s descendants settle the cosmos. Expanding faster (e.g. by achieving a faster travel speed of 76%c instead of 75%c) could then count as a speed-up. That said, it is difficult for this to be a contingent speed-up, as it raises the question of why (when this was the main determinant of value) would this improvement not be implemented at a later date? And it is also difficult in this case to see how any actions now could produce such a speed-up.
Overall, I don’t see them as a promising avenue for current efforts to target, and more of a useful theoretical tool, but I also don’t think there are knockdown arguments against them being a practical avenue.