Non-EA interests include chess and TikTok (@benthamite). We are probably hiring: https://ââmetr.org/ââhiring
Ben_Westđ¸
Karma: 14,902
Those seem like reasonable categories. I disagree with the antecedent but will rephrase your question as âwhat type of decline am I most worried about?â, the answer to which is â1âł.
In particular: I think itâs possible that AI will be such a compelling issue that the intellectual excitement and labor moves away from cause-neutral EA to EA-flavored AI Safety. This is already happening a bit (and also happened with animal welfare); it doesnât seem crazy to think that it will become a larger problem.
Thanks for cross posting these blogs to the forum. You are one of my favorite new (to me) EA writers, and I probably wouldnât see your work as much if it wasnât cross posted.
Thanks for writing this, I think itâs interesting to read case studies like these.
Recovery is not likely historically
As with many âp->qâ arguments, I think this one is true for the trivial reason that q holds independent of p. I.e. itâs true that itâs unlikely that people will identify as EAâs one hundred years from now[1] but that also was unlikely 5 years ago.
I asked Claude âWhat are five randomly chosen social movements which started at least 100 years ago? Give no regard to whether they still currently exist.â and it said:
The Temperance Movement (1820s-1930s) - A campaign to reduce or eliminate alcohol consumption, which gained significant momentum in the United States and other countries. It ultimately led to Prohibition in the US from 1920-1933.
The Suffragette Movement (1840s-1920s) - The organized effort to secure voting rights for women, with major campaigns in Britain, the United States, and other countries. It employed tactics ranging from peaceful protests to more militant actions.
The Abolitionist Movement (1830s-1870s) - The campaign to end slavery, particularly prominent in the United States and Britain. It included figures like Frederick Douglass and William Lloyd Garrison and contributed to the American Civil War.
The Eight-Hour Day Movement (1860s-1880s) - Labor activism focused on establishing an eight-hour working day, with major events like the Haymarket Affair in Chicago in 1886. It was part of broader labor organizing efforts worldwide.
The Theosophical Movement (1875-early 1900s) - A spiritual and philosophical movement founded by Helena Blavatsky and others, blending Eastern and Western religious concepts. It promoted ideas about universal brotherhood and ancient wisdom traditions.
None of these still exist, though three (suffragette, abolitionist, eight-hour day) ~accomplished their goals which maybe you want to exclude as a reason for movements to stop existing. So I asked for three more and it said:
The Back-to-Africa Movement (1890s-1920s) - Led by figures like Marcus Garvey and his Universal Negro Improvement Association, this movement encouraged people of African descent to return to Africa and promoted Black nationalism and pride. It was particularly influential in the early 20th century.
The Settlement House Movement (1880s-1920s) - A social reform effort where educated volunteers lived in poor urban neighborhoods to provide services and advocate for better conditions. Jane Addamsâ Hull House in Chicago was one of the most famous examples.
The Eugenics Movement (1900s-1940s) - A pseudoscientific social movement that promoted âimprovingâ human heredity through selective breeding and sterilization programs. While thoroughly discredited today, it was influential in early 20th century policy in many countries including the United States and Germany.
So the base rate for lasting 100+ years is either 0â8, 0â5, or 3â8, depending on how you want to classify the movements which accomplished their goals. This does not seem far off from the 1â5 success rate in your data set.
- ^
I roughly mentally tried to average the age of your case studies and thought that they were about 100 years old on average, but this isnât precise.
- ^
Some things you can donate to:
PauseAI (not the same as PauseAI US, despite the name)
Fixed the link. I also tried your original prompt and it worked for me.
But interesting! The âHarder word, much vaguer clueâ seems to prompt it to not actually play hangman and instead antagonistically try to post hoc create a word after each guess which makes your guess wrong. I asked âDid you come up with a word when you first told me the number of letters or are you changing it after each guess?â And it said âI picked the word up front when I told you it was 10 letters long, and I havenât changed it since. Youâre playing against that same secret word the whole time.â (Despite me being able to see its reasoning trace that this is not what itâs doing.) When I say I give up it says âIâm sorryâI actually lost track of the word Iâd originally picked and canât accurately reveal it now.â (Because it realized that there was no word consistent with its clues, as you noted.)
So I donât think itâs correct to say that it doesnât know how to play hangman. (It knows, as you noted yourself.) It just wants so badly to make you lose that it lies about the word.
Huh interesting, I just tried that direction and it worked fine as well. This isnât super important but if you wanted to share the conversation Iâd be interested to see the prompt you used.
By analogy, o4-miniâs inability to play hangman is a sign that itâs far from artificial general intelligence (AGI)
What is your source for this? I just tried and it played hangman just fine.
Given that some positions in EA leadership are already elected, I might suggest changing the wording to something like:
There should be an international body whose power is roughly comparable to CEA whose leadership is elected
I think I agree with your overall point but some counterexamples:
EA Criticism and Red Teaming Contest winners. E.g. GiveWell said âWe believe HLIâs feedback is likely to change some of our funding recommendations, at least marginally, and perhaps more importantly improve our decision-making across multiple interventionsâ
GiveWell said of their Change Our Mind contest âTo give a general sense of the magnitude of the changes we currently anticipate, our best guess is that Matthew Romer and Paul Romer Presentâs entry will change our estimate of the cost-effectiveness of Dispensers for Safe Water by very roughly 5 to 10% and that Noah Haberâs entry may lead to an overall shift in how we account for uncertainty (but itâs too early to say how it would impact any given intervention).â
HLI discussed some meaningful ways they changed as the result of criticism here.
This is an extremely helpful response, thank you!
This is cool, I like BHAGs in general and this one in particular. Do you have a target for when you want to get to 1M pledgers?
If you manage to convince an investor that timelines are very short without simultaneously convincing them to care a lot about x-risk, I feel like their immediate response will be to rush to invest briefcases full of cash into the AI race, thus helping make timelines shorter and more dangerous.
Iâm the corresponding author for a paper that Holly is maybe subtweeting and was worried about this before publication but donât really feel like those fears were realized.
Firstly, I donât think there are actually very many people who sincerely think that timelines are short but arenât scared by that. I think what you are referring to is people who think âtimelines are shortâ means something like âAI companies will 100x their revenue in the next five yearsâ, not âAI companies will be capable of instituting a global totalitarian state in the next five years.â There are some people who believe the latter and arenât bothered by it but in my experience they are pretty rare.
Secondly, when VCs get the âAI companies will 100x their revenue in the next five yearsâ version of short timelines they seem to want to invest into LLM-wrapper startups, which makes sense because almost all VC firms lack the AUM to invest in the big labs.[1] I think there are plausible ways in which this makes timelines shorter and more dangerous but it seems notably different from investing in the big labs.[2]
Overall, my experience has mostly been that getting people to take short timelines seriously is very close to synonymous with getting them to care about AI risk.
- ^
Caveat that ~everyone has the AUM to invest in publicly traded stocks. I didnât notice any bounce in share price for e.g. NVDA when we published and would be kind of surprised if there was a meaningful effect, but hard to say.
- ^
Of course, thereâs probably some selection bias in terms of who reaches out to me. Masayoshi Son probably feels like he has better info than what I could publish, but by that same token me publishing stuff doesnât cause much harm.
- ^
Do you think that distancing is ever not in the interest of both parties? If so, what is special about Anthropic/âEA?
(I think itâs plausible that the answer is that distancing is always good; the negative risks of tying your reputation to someone always exceed the positive. But Iâm not sure.)
Thanks for doing this Saulius! I have been wondering about modeling the cost effectiveness of animal welfare advocacy under assumptions of relatively short AI timelines. It seems like one possible way of doing this is to to change the âYearly decrease in probability that commitment is relevantâ numbers in your sheet (cells I28:30). Do you have any thoughts on that approach?
You had never thought through âwhether artificial intelligence could be increasing faster than Mooreâs law.â Should we conclude that AI risk skeptics are âinsular, intolerant of disagreement or intellectual or social non-conformity (relative to the groupâs norms), and closed-off to even reasonable, relatively gentle criticism?â
I have to say, the bad part supports my observation!
Steven was responding to this:
The community of people most focused on keeping up the drumbeat of near-term AGI predictions seems insular, intolerant of disagreement or intellectual or social non-conformity (relative to the groupâs norms), and closed-off to even reasonable, relatively gentle criticism
None of Stevenâs bullet points support this. Many of them say the exact opposite of this.
More seriously, I didnât really think through precisely whether artificial intelligence could be increasing faster than Mooreâs law.
Fair enough, but in that case I feel kind of confused about what your statement âProgress does not seem like a fast exponential trend, faster than Mooreâs lawâ was intended to imply.
If the claim you are making is âAGI by 2030 will require some growth faster than Mooreâs lawâ then the good news is that almost everyone agrees with you but the bad news is that everyone already agrees with you so this point is not really cruxy to anyone.
Maybe you have an additional claim like â...and growth faster than mooreâs law is unlikely?â If so, I would encourage you to write that because I think that is the kind of thing that would engage with peopleâs cruxes!
If you drew a chart for the GPT models on ARC-AGI-2, it would mostly just be a flat line.. Itâs only with the o3-low and o1-pro models we see scores above 0%
⌠which is what (super)-exponential growth looks like, yes?
Specifically: Weâve gone from o1 (low) getting 0.8% to o3 (low) getting 4% in ~1 year, which is ~2 doublings per year (i.e. 4x Mooreâs law). Forecasting from this few data points sure seems like a cursed endeavor to me, but if you want to do it then I donât see how you can rule out Mooreâs-law-or-faster growth.
I would be curious to know what the best benchmarks are which show a sub-Mooreâs-law trend.
Congrats to the team on the metrics stabilizing and also congrats Sarah on the promotion![1]
Also maybe congrats to Toby, although I canât tell if the new title is a promotion?