Unable to work. Was community director of EA Netherlands, had to quit due to long covid. Everything written since 2021 with considerable brain fog, and bad at maintaining discussions since.
I have a background in philosophy, risk analysis, and moral psychology. I also did some x-risk research. Currently most worried about AI and US democracy. (Regarding the latter, I’m highly ranked on Manifold).
It’s here!
Ah yeah that seems fine then! “Life” is an imprecise term and I’d prefer “sentience” or “sentient beings” but maybe I’m overdoing it
For forecasts, here’s Manifold’s US Democracy questions, which I suggest sorting by total traders (and unfortunately, anything n<30 traders becomes quite unreliable) and I also have a Manifold dashboard compiled where questions are grouped a bit more by theme here.
Main questions are:
“If Trump is elected, will the US still be a liberal democracy at the end of his term? (58%, n = 191)”—criticism of the V-DEM benchmark here
“Will the United States experience a constitutional crisis before 2030? (73%, n = 123)”
“Will Donald Trump arrest his political opponents [before 2026]? (41%, n = 76)”
“Will a sitting US President refuse to follow or ignore a Supreme Court ruling by 2032? (55%, n = 68)”
“Will Donald Trump remain de facto leader of the United States beyond the end of his second term? (7%, n = 44)”
I really like that you’ve chosen this topic and think it’s an important one! I wrote my MA Philosophy thesis on this (in 2019, now outdated).
On the margin[1], it is better to work on reducing the chance of our[2] extinction than increasing the value of futures where we survive
I want to flag that I disagree with this framing, as it’s very anthropocentric. There are futures in which we go extinct but that are nevertheless highly valuable (happy sentient AI spreading via VNM probes). Perhaps more empirically relevant, I expect almost all effects to go via making the transition to superintelligence go well, and the most distinct action is focusing on digital sentience (which has little effect on extinction risk and much effect on the value of the future).
Perhaps it would be interesting to run a one-time survey on the political beliefs in our community? (and how they compare to the general population)
(I think this should be done on the level of semi-abstract questions and not hot topic political issues)
I still think using a single dimension for political beliefs is losing out on a lot of relevant information—I think our community is more diverse than implied by these results.
What about AISIs? They might have access and incentive?
Do you think any of the frontier companies will care intrinsically about the sentience of their (future) models?
Here’s a few areas that might be affected, per Zack Beauchamp from Vox:
Less reliable nuclear umbrella → higher risk of nuclear proliferation
Retraction from NATO & eroding the resurrect for territorial sovereignty → higher risk of territorial expansion wars
Less invested in global infectious disease control → higher risk of pandemics
Trade wars, economic instability → global economic slowdown
How about adding this one?
https://manifold.markets/Siebe/if-trump-is-elected-will-the-us-sti
The Malaysia prevalence is based on a small sample size of n = 595 (from the Fishera et al. Meta-analysis) so it would suggest leaving that out
A study of 53 patients who had used psilocybin or LSD found that “twenty-two of 26 psilocybin users reported that psilocybin aborted attacks; 25 of 48 psilocybin users and 7 of 8 LSD users reported cluster period termination; 18 of 19 psilocybin users and 4 of 5 LSD users reported remission period extension” (Sewell et al., 2006). Another online survey of 270 cluster headache sufferers revealed that 68% of respondents who used tryptamines had a 4 or 5 out of 5 relief, with 5 being “completely eliminated the cluster headaches” (Frerichs, 2019).
While these are very promising effect sizes, the methods are of low quality (e.g. high risk of bias). I would suggest a properly powered RCT before concluding that psychedelics are really that effective.
I just learned that Lawrence Lessig, the lawyer who is/was representing Daniel Kokateljo and other OpenAI employees, supported and encouraged electors to be faithless and vote against Trump in 2016.
He wrote an opinion piece in the Washington Post (archived) and offered free legal support. The faithless elector story was covered by Politico, and was also supported by Mark Ruffalo (the actor who recently supported SB-1047).
I think this was clearly an attempt to steal an election and would discourage anyone from working with him.
I expect someone to eventually sue AGI companies for endangering humanity, and I hope that Lessig won’t be involved.
Thanks for doing this!
I don’t know how useful the results are, as extinction is not the only existentially catastrophic scenario I care about. And I wonder if and how the ranking changes when the question is about existential catastrophe. For example, do people think AI is unlikely to cause extinction but likely to cause a bad form of human disempowerment?
Nice list! I often also see Art of the Gathering as recommendation but it didn’t make yours?
Drunk driving is illegal because it risks doing serious harm. It’s still illegal when the harm has not occurred (yet). Things can be crimes without harm having occurred.
I guess this is the same dynamic as why movie and sports stars are high status in society: they are highly visible compared to more valuable members of society (and more entertaining to watch). We don’t really see much of highly skilled operations people compared to researchers
That seems relevant for AI vs. Humans, but not for AI vs AI.
Most totalitarian regimes are pretty bad at creating value, with China & Singapore as exceptions. (But in many regimes, creating that value isn’t necessary to remain in power of there’s e.g. income from oil)
Here’s one idea as a reference:
The campaign for ranked choice voting-style reforms in 7 states cost nearly $100 million, and failed in 6 out of 7 states (it was narrowly protected in Alaska) in 2024. The linked article is a decent description.
Ranked Choice Voting is a good way to reduce polarization in politics, elect more popular (and less extreme) candidates, and increase competition. It would also reduce the power of Trump over the Republican Party, which could lead to more Congressional pushback.
Despite the disappointing 2024 results, I believe there significant opportunity in 2026: the midterms have a more politically engaged turnout, and given the current situation, voters might be more open to reform.
There are alternatives to Ranked Choice Voting (like approval voting) and I’m no expert on them. It does seem like, if this campaign were to run for 2026 it would need to start soon.
There’s also a decent chance that it would be perceived as very hostile by the current administration, and retaliation could do significant damage to the community or the specific funders behind a campaign