«If we are the only rational beings in the Universe, as some recent evidence suggests, it matters even more whether we shall have descendants or successors during the billions of years in which that would be possible. Some of our successors might live lives and create worlds that, though failing to justify past suffering, would have given us all, including those who suffered most, reasons to be glad that the Universe exists.» — Derek Parfit
Pablo_Stafforini
Karma: 3,112
AGI Predictions
In some cases Trump has been bad, but for the opposite reason than you were worried about! For example you criticized him for supporting travel bans during Ebola
It’s not the opposite reason. The underlying criticism is that Trump’s measures were miscalibrated to the magnitude of the problem. If your decision-making process is deeply flawed, as Trump’s is, you should expect miscalibration in both directions.
Leopold has now published a popular article discussing this topic. Highly recommended.
An excerpt:
Philosophers like Nick Bostrom, Derek Parfit, and Toby Ord have become increasingly concerned about such so-called “existential risks.” An unrecoverable collapse of civilization wouldn’t just be tragic for the billions who would suffer and die. Perhaps the greatest tragedy would be the foreclosing of all of humanity’s potential. Humanity could flourish for billions of years and enable trillions of happy human lives—if only we do not destroy ourselves beforehand.
This line of thinking has led some to question whether “progress”—in particular, technological progress—is as straightforwardly beneficial as commonly assumed. Nick Bostrom imagines the process of technological development as “pulling balls out of a giant urn.” So far, we’ve been lucky, pulling out a great many “white” balls that are broadly beneficial. But someday, we might pull out a “black” ball: a new technology that destroys humanity. Before that first nuclear test, some of the physicists worried that the nuclear bomb would ignite the atmosphere and end the world. Their calculations ultimately deemed it “extremely unlikely,” and so they proceeded with the test—which, as it turns out, did not end the world. Perhaps the next time, we don’t get so lucky.
The same technological progress that creates these risks is also what drives economic growth. Does that mean economic growth is inherently risky? Economic growth has brought about extraordinary prosperity. But for the sake of posterity, must we choose safe stagnation instead? This view is arguably becoming ever-more popular, particularly amongst those concerned about climate change; Greta Thunberg recently denounced “fairy tales of eternal economic growth” at the United Nations.
I argue that the opposite is the case. It is not safe stagnation and risky growth that we must choose between; rather, it is stagnation that is risky and it is growth that leads to safety.
We might indeed be in “time of perils”: we might be advanced enough to have developed the means for our destruction, but not advanced enough to care sufficiently about safety. But stagnation does not solve the problem: we would simply stagnate at this high level of risk. Eventually, a nuclear war or environmental catastrophe would doom humanity regardless.
Faster economic growth could initially increase risk, as feared. But it will also help us get past this time of perils more quickly. When people are poor, they can’t focus on much beyond ensuring their own livelihoods. But as people grow richer, they start caring more about things like the environment and protecting against risks to life. And so, as economic growth makes people richer, they will invest more in safety, protecting against existential catastrophes. As technological innovation and our growing wealth has allowed us to conquer past threats to human life like smallpox, so can faster economic growth, in the long run, increase the overall chances of humanity’s survival.
I sympathise with this, but I think if we don’t have public posts like this one, the outcome is more-or-less decided in advance.
Yes, I agree. What I’m uncertain about is whether it’s desirable to have more of these posts at the current margin. And to be clear: by saying I’m uncertain whether it’s a good idea, I don’t mean to suggest it’s not a good idea; I’m simply agnostic.
This comment expresses something I was considering saying, but more clearly than I could. I would add that thinking strategically about this cultural phenomenon involves not only trying to understand its mechanism of action, but also coming up with frameworks for deciding what tradeoffs to make in response to it. I am personally very disturbed by the potential of cancel culture to undermine or destroy EA, and my natural reaction is to believe that we should stand firm and make no concessions to it, as well as to upvote posts and comments that express this sentiment. This is not, however, a position I feel I can endorse on reflection: it seems instead that protecting our movement against this risk involves striking a difficult and delicate balance between excessive and insufficient relaxation of our epistemic standards. By giving in too much the EA movement risks relinquishing its core principles, but by giving in too little the movement risks ruining its reputation. Unfortunately, I suspect that an open discussion of this issue may itself pose a reputational risk, and in fact I’m not sure it’s even a good idea to have public posts like the one this comment is responding to, however much I agree with it.
On a charitable reading of Parfit, the ‘muzak and potatoes’ expression is meant to pick out the kind of phenomenal experience associated with the “drab existence” he wants to communicate to the reader. So he is not asking you to imagine a life where you do nothing but listen to muzak and eat potatoes. Instead, he is asking you to consider how it typically feels like to listen to muzak and eat potatoes, and to then imagine a life that feels like that, all the time.
On the one hand, my opinion of Metaculus predictions worsened as I saw how the ‘recent predictions’ showed people piling in on the median on some questions I watch.
Can you say more about this? I ask because this behavior seems consistent with an attitude of epistemic deference towards the community prediction when individual predictors perceive it to be superior to what they can themselves predict given their time and ability constraints.
The US democracy may be at risk. It is only “our democracy” for 4.25% of the world’s population.
(Apologies for focusing on a single word of your post, but I think this seemingly trivial semantic difference reflects a more substantive and widespread issue. How many concerned posts about the political situation in, say, India have you seen in the Forum recently? How many “action items” for protecting democracy in, say, Brazil have you encountered? It is depressing and, yes, irritating to see a community that supposedly values all people equally concentrate their attention so overwhelmingly on a single country when it comes to politics and “current affairs”.)
UK to host human challenge trials for Covid-19 vaccines
I agree that the sentence Linch quoted sounds like a “bravery debate” opening, but that’s not how I perceive it in the broader context. I don’t think the author is presenting himself/herself as an underdog, intentionally or otherwise. Rather, they are making that remark as part of their overall attempt to indicate that they are aware that they are raising a sensitive issue and that they are doing so in a collaborative spirit and with admittedly limited information. This strikes me as importantly different from the prototypical bravery debate, where the primary effect is not to foster an atmosphere of open dialogue but to gain sympathy for a position.
I am tentatively in agreement with you that “clarification of intent” can be done without “bravery talk”, by which I understand any mention that the view one is advancing is unpopular. But I also think that such talk doesn’t always communicate that one is the underdog, and is therefore not inherently problematic. So, yes, the OP could have avoided that kind of language altogether, but given the broader context, I don’t think the use of that language did any harm.
(I’m maybe 80% confident in what I say above, so if you disagree, feel free to push me.)
Thanks, you are right. I have amended the last sentence of my comment.
FWIW, I think that the qualification was very appropriate and I didn’t see the author as intending to start a “bravery debate”. Instead, the purpose appears to have been to emphasize that the concerns were raised in good faith and with limited information. Clarifications of this sort seem very relevant and useful, and quite unlike the phenomenon described in Scott’s post.
The link is broken; can you fix it?
In the meantime, a few random thoughts. First, the index fund analogy suggests a self-correcting mechanism. Players defer to the community only to the degree that they expect it to track the truth more reliably than their individual judgment, given their time and ability constraints. As the reliability of the community prediction changes, in response to changes in the degree to which individual players defer to it, so will these players’s willingness to defer to the community.
Second, other things equal, I think it’s a desirable property of a prediction platform that it makes it rational for players to sometimes defer to the community. This could be seen as embodying the important and neglected truth that in many areas of life one can generally do better by deferring to society’s collective wisdom than by going with one’s individual opinion. Furthermore, it requires considerable ability to determine when and to what degree one should defer to others in any given case. In fact, this metacognitive skill of knowing how much more (or less) reliable other opinions are relative to one’s own seems like a core epistemic virtue, and one that can be assessed only if users are allowed to defer to others.
Finally, insofar as there are reasons for wanting players not to defer to the community, I think the appropriate response is to change the scoring function rather than to ask players to exercise self-restraint. As fellow forecaster Tom Adamczewski reminded me, the Metaculus Scoring System page describes one such possible change:
It’s easy to account for the average community prediction by adding a constant to each of these. For example, . This way a player would get precisely zero points if they just go along with the community average.
Perhaps Metaculus could have two separate leaderboards: in addition to the current ranking, it could also display a ranking of players with the community component subtracted. These two rankings could be seen as measuring the quality of a player’s “credences” and “impressions”, respectively.
My spacemacs config file is here. The main Keyboard Maestro macros I use are here. As noted, these macros were created back when I was beginning to use Emacs, so they don’t make use of org capture or other native functionality (including Emacs own internal macros, or the even more powerful elmacro package). I plan to review these files at some point, but not in the immediate future. Happy to answer questions if anything is unclear.
Yes, indeed. I was about to suggest an edit to the transcript to make that clear. When I created the Keyboard Maestro script, I was still relatively unfamiliar with Org mode so I didn’t make use of org capture. But that’s the proper way to do it.
It was a pleasure to discuss my approach to forecasting with Jungwon and Amanda. I’d be happy to clarify anything that I failed to explain properly during our conversation, or to answer any questions related to the implementation or reasoning behind my “system” (if one may call it that).
I haven’t watched the documentary, but I’m antecedently skeptical of claims that social media constitute an existential risk in the sense in which EAs use that term. The brief summary provided by the Wikipedia article doesn’t seem to support that characterization:
the film explores the rise of social media and the damage it has caused to society, focusing on its exploitation of its users for financial gain through surveillance capitalism and data mining, how its design is meant to nurture an addiction, its use in politics, its impact on mental health (including the mental health of adolescents and rising teen suicide rates), and its role in spreading conspiracy theories and aiding groups such as flat-earthers and white supremacists.
While many of these effects are terrible (and concern about them partly explains why I myself basically don’t use social media), they do not appear to amount to threats of existential catastrophe. Maybe the claim is that the kind of surveillance made possible by social media and big tech firms more generally (“surveillance capitalism”) has the potential to establish an unrecoverable global dystopia?
Are there other concrete mechanisms discussed by the documentary?
To what degree are the differences between longtermists who prioritize s-risks and longtermists who prioritize x-risks driven by moral disagreements about the relative importance of suffering versus happiness, rather than by factual disagreements about the relative magnitude of s-risks versus x-risks?
The universe is vast, so it seems there is a lot of room for variation even within the subset of risks involving astronomical quantities of suffering. How much, in your opinion, do s-risks vary in severity? Relatedly, what are your grounds for singling out s-risks as the object of concern, rather than those risks involving the most suffering?
Together with a few EA friends, I ended up betting a substantial amount of money on Biden. It went well for me, too, as well as for some of my friends. I think presidential elections present unusually good opportunities for both betting and arbitrage, so it may be worth coordinating some joint effort next time.
(As a note of historical interest, during the 2012 US election a small group of early EAs made some money arbitraging Intrade.)