Great idea Thomas.
I’ve just sent a letter and encourage others to do so too!
Toby_Ord
Karma: 3,175
A small correction:
Infamously there was a period where some scientists on the project were concerned that a nuclear bomb would ignite the upper atmosphere and end all life on Earth; fortunately they were able to do some calculations suggesting that showed beyond reasonable doubt that this would not happen before the Trinity test occurred.
The calculations suggesting the atmosphere couldn’t ignite were good, but were definitively not beyond reasonable doubt. Fermi and others kept working to re-check the calculations in case they’d missed something all the way up to the day of the test and wouldn’t have done so if they were satisfied by the report.
The report (published after Trinity) does say:
One may conclude that the arguments of this paper make it unreasonable to expect that the N + N reaction could propagate. An unlimited propagation is even less likely.
That is often quoted by people who want to suggest the case was closed, but the next (and final) sentence of the report says:
However, the complexity of the argument and the absence of satisfactory experimental foundations makes further work on the subject highly desireable.
Great piece William — thanks for sharing it here.
I liked your strategy for creating robust principles that would have worked across a broad range of cases, and it would be good to add others to the Manhattan Project example.
I particularly liked you third principle:
Principle 3: When racing, have an exit strategy
In the case of the Manhattan project, a key moment was the death of Hitler and surrender of Germany. Given that this was the guiding reason — the greater good with which the scientists justify their creation of a terrible weapon — it is very poor how little changed at that point. Applying your principles, one could require a very special meeting if/when any of the race-justifying conditions disappear, to force reconsideration at that point.
This paragraph was intended to speak to the relevance of this argument given that (as you say) we can’t easily advance all progress uniformly:
And it may have some uncomfortable consequences. If advancing all progress would turn out to be bad, but advancing some parts of it would be good, then it is likely that advancing the remaining parts would be even more bad. Since some kinds of progress are more plausibly linked to bringing about an earlier demise (e.g. nuclear weapons, climate change, and large-scale resource depletion only became possible because of technological, economic, and scientific progress) these parts may not fare so well in such an analysis. So it may really be an argument for differentially boosting other kinds of progress, such as moral progress or institutional progress, and perhaps even for delaying technological, economic, and scientific progress.
Thanks Mike — a very useful correction. I’m genuinely puzzled as to why this didn’t lead to a more severe early response given China’s history with SARS. That said, I can’t tell from the article how soon the sample from this patient was sequenced/analysed.
Yes, that’s right about the track-skipping condition for the exogenous case, and I agree that there is a strong case the end of factory farming will be endogenous. I think it is a good sign that the structure of my model represents some/all of the key considerations in your take on progress too — but with the different assumption about the current value changing the ultimate conclusion.
I delivered this talk before the Rootclaim debate, though I haven’t followed that debate since, so can’t speak to how much it has changed views. I was thinking of the US intelligence community’s assessments and the diversity of opinions among credible people who’ve looked into it in detail.
I meant vaccines for diseases that didn’t yet have a vaccine. The 1957 case was a vaccine for a new strain of influenza, when they already had influenza vaccines.
That’s an interesting and unusual argument for progress:
Progress so far has brought us to a point where we are causing so much harm on a global scale that the value of each year is large and negative. But pushing on further with progress is a good thing because it will help end this negative period.
That could well be correct, though it is also very different from the usual case by proponents of progress.
I hadn’t seen that and I agree that it looks like a serious negative update (though I don’t know what exactly it is measuring). Thanks for drawing it to my attention. I’m also increasingly worried about the continued unprecedentedly hot stretch we are in. I’d been assuming it was just one of these cases of a randomly hot year that will regress back to the previous trend, but as it drags on the hypothesis of there being something new happening does grow in plausibility.
Overall, ‘mixed’ might be a better summary of Climate.
The Precipice Revisited
The hypothetical being considered in this piece is that all progress is advanced by a year. So e.g. we have everything humanity had in the year 1000 in the year 999 instead. Imagine that it was literally exactly the same state of the world, but achieved a year earlier. Wouldn’t we then expect to have 2025 technology in 2024? If not, what could be making the effect go away?
In general, there is not much of an external clock that would be setting quality of life in the year 3000 just because that is what the calendar says. It will mainly depend on the internal clock of the attributes of civilisation and this is the clock we are imagining advancing.
This is very different to the the global health and wellbeing interventions, which I wouldn’t class as attempting to ‘advance progress’ and where I’d be surprised to see a significant permanent effect. The permanence will be stronger if the effect is broader across different sectors, or if it is in a sector that is driving the others, and it is aimed at the frontiers of knowledge or technology or institutions, rather than something more like catch-up growth.
Yes, it could be interesting to try to understand the most plausible trajectory-altering things like this that will occur on the exogenous calendar time. They could be very important.
Interesting posts!
I don’t recall reading either of them before.
Paul’s main argument in the second piece is that progress “doesn’t have much effect on very long-term outcomes” because he expects this quality-of-life curve to eventually reach a plateau. That is a somewhat different argument. As I show in Shaping Humanity’s Longterm Trajectory, the value of advancements can be extremely large as they scale with the instantaneous value of the future. e.g. it is plausible that if civilisation has spread to billions of billions of worlds that having an extra year of this is worth a lot (indeed this is the point of the first part of ‘Astronomical Waste’). But it doesn’t scale with the duration of our future, so can get beaten by things like reducing existential risk that scale with duration as well.
So the argument that we are likely to reach a plateau is really an argument that existential risk (or some other trajectory changes) will be even more important than progress, rather than that progress may be as likely to be negative value as positive value. For that you need to consider that the curve will end and that the timing of this could be endogenous.
(That said, I think arguments based on plateaus could be useful and powerful in this kind of longtermist reasoning, and encourage people to explore them. I don’t think it is obvious that we will reach a plateau (mainly as we might not make it that far), but they are a very plausible feature of the trajectory of humanity that may have important implications if it exists, and which would also simplify some of the analysis.)
On the Value of Advancing Progress
To be clear, I don’t think that most things are longtermist interventions with these permanent impacts (and most things aren’t trying to be).
Clearly some thing have had lasting effects. e.g. there were no electrical household items before the 19th century, and now they are ubiquitous and quite possibly will be with us as long as humanity lasts. Whereas if humanity had never electrified, the present would be very different.
That said, it is also useful to ask about the counterfactuals. e.g. if Maxwell or Edison or any other pioneer hadn’t made their discoveries, how different would we expect 2024 to be? In this case, it is less clear as someone else probably would have made these discoveries later. But then discoveries that build on them would have been delayed etc. I doubt that the counterfactual impact of Maxwell goes to zero until such a point as practically everything has been discovered (or everything downstream of electromagnetism). That said, it could easily have diminishing impact over time, as is typical of advancements (their impact does not scale with humanity’s duration).
But note that the paper actually doesn’t claim that a particular invention or discovery acts as an advancement. It suggests it is a possible model of the longterm impacts of things like that. One reason I bring it up is that it shows that even if it was a permanent advancement, then under a wide set of circumstances, it would still be beaten by reducing existential risk. And even if attempts to advance progress really were lasting advancements, the value could be negative if they also bring forward the end time.
I would love it if there was more investigation of the empirical measurement of such lasting effects, though it is outside of my field(s).
Thanks for this excellent writeup of a very promising project.
A somewhat useful comparator would be to the proportion of people in Wuhan who were infected at the time that SARS-CoV 2 was detected. According to this report, the pathogen was identified on Jan 8, and by Jan 18, their estimate is that 2,500 – 6,100 of the 11 million people were infected (they don’t seem to report a number for Jan 8, though you might be able to work it out from their doubling time results). That is about 0.02% – 0.06%. So it sounds like you would need to scale this up by a factor of 20 or so to detect it as quickly as Wuhan detected their non-stealth corona virus.
Of course, SARS-CoV 2 wouldn’t have been found anywhere near as fast if it were a stealth pathogen, so this comparison only goes so far, but it is a start.
I really like your framing of “making doing good your central purpose”.
I also like the way the OP shows how we can embrace and support people of different levels of dedication, but I wonder whether having binary names (especially identity group names) is a good idea. Ultimately this is a spectrum (without the clear delineation and identity marker that comes in religions with monks) and it is probably good to have terms that reflect the continuum nature of it.
I discuss something similar to that a bit on page 41, but mainly focusing on whether depletion could make it harder for civilisation to re-emerge. Ultimately, it still looks to me like it would be easier and faster the second time around.
Thank you so much for everything you’ve done. You brought such renewed vigour and vision to Giving What We Can that you ushered it into a new era. The amazing team you’ve assembled and culture you’ve fostered will put it such good stead for the future.
I’d strongly encourage people reading this to think about whether they might be a good choice to lead Giving What We Can forward from here. Luke has put it in a great position, and you’d be working with an awesome team to help take important and powerful ideas even further, helping so many people and animals, now and across the future. Do check that job description and consider applying!