Non-EA interests include chess and TikTok (@benthamite). We are probably hiring: https://ââmetr.org/ââhiring
Ben_Westđ¸
Karma: 14,872
Huh interesting, I just tried that direction and it worked fine as well. This isnât super important but if you wanted to share the conversation Iâd be interested to see the prompt you used.
By analogy, o4-miniâs inability to play hangman is a sign that itâs far from artificial general intelligence (AGI)
What is your source for this? I just tried and it played hangman just fine.
Given that some positions in EA leadership are already elected, I might suggest changing the wording to something like:
There should be an international body whose power is roughly comparable to CEA whose leadership is elected
I think I agree with your overall point but some counterexamples:
EA Criticism and Red Teaming Contest winners. E.g. GiveWell said âWe believe HLIâs feedback is likely to change some of our funding recommendations, at least marginally, and perhaps more importantly improve our decision-making across multiple interventionsâ
GiveWell said of their Change Our Mind contest âTo give a general sense of the magnitude of the changes we currently anticipate, our best guess is that Matthew Romer and Paul Romer Presentâs entry will change our estimate of the cost-effectiveness of Dispensers for Safe Water by very roughly 5 to 10% and that Noah Haberâs entry may lead to an overall shift in how we account for uncertainty (but itâs too early to say how it would impact any given intervention).â
HLI discussed some meaningful ways they changed as the result of criticism here.
This is an extremely helpful response, thank you!
This is cool, I like BHAGs in general and this one in particular. Do you have a target for when you want to get to 1M pledgers?
If you manage to convince an investor that timelines are very short without simultaneously convincing them to care a lot about x-risk, I feel like their immediate response will be to rush to invest briefcases full of cash into the AI race, thus helping make timelines shorter and more dangerous.
Iâm the corresponding author for a paper that Holly is maybe subtweeting and was worried about this before publication but donât really feel like those fears were realized.
Firstly, I donât think there are actually very many people who sincerely think that timelines are short but arenât scared by that. I think what you are referring to is people who think âtimelines are shortâ means something like âAI companies will 100x their revenue in the next five yearsâ, not âAI companies will be capable of instituting a global totalitarian state in the next five years.â There are some people who believe the latter and arenât bothered by it but in my experience they are pretty rare.
Secondly, when VCs get the âAI companies will 100x their revenue in the next five yearsâ version of short timelines they seem to want to invest into LLM-wrapper startups, which makes sense because almost all VC firms lack the AUM to invest in the big labs.[1] I think there are plausible ways in which this makes timelines shorter and more dangerous but it seems notably different from investing in the big labs.[2]
Overall, my experience has mostly been that getting people to take short timelines seriously is very close to synonymous with getting them to care about AI risk.
- ^
Caveat that ~everyone has the AUM to invest in publicly traded stocks. I didnât notice any bounce in share price for e.g. NVDA when we published and would be kind of surprised if there was a meaningful effect, but hard to say.
- ^
Of course, thereâs probably some selection bias in terms of who reaches out to me. Masayoshi Son probably feels like he has better info than what I could publish, but by that same token me publishing stuff doesnât cause much harm.
- ^
Do you think that distancing is ever not in the interest of both parties? If so, what is special about Anthropic/âEA?
(I think itâs plausible that the answer is that distancing is always good; the negative risks of tying your reputation to someone always exceed the positive. But Iâm not sure.)
Thanks for doing this Saulius! I have been wondering about modeling the cost effectiveness of animal welfare advocacy under assumptions of relatively short AI timelines. It seems like one possible way of doing this is to to change the âYearly decrease in probability that commitment is relevantâ numbers in your sheet (cells I28:30). Do you have any thoughts on that approach?
You had never thought through âwhether artificial intelligence could be increasing faster than Mooreâs law.â Should we conclude that AI risk skeptics are âinsular, intolerant of disagreement or intellectual or social non-conformity (relative to the groupâs norms), and closed-off to even reasonable, relatively gentle criticism?â
I have to say, the bad part supports my observation!
Steven was responding to this:
The community of people most focused on keeping up the drumbeat of near-term AGI predictions seems insular, intolerant of disagreement or intellectual or social non-conformity (relative to the groupâs norms), and closed-off to even reasonable, relatively gentle criticism
None of Stevenâs bullet points support this. Many of them say the exact opposite of this.
More seriously, I didnât really think through precisely whether artificial intelligence could be increasing faster than Mooreâs law.
Fair enough, but in that case I feel kind of confused about what your statement âProgress does not seem like a fast exponential trend, faster than Mooreâs lawâ was intended to imply.
If the claim you are making is âAGI by 2030 will require some growth faster than Mooreâs lawâ then the good news is that almost everyone agrees with you but the bad news is that everyone already agrees with you so this point is not really cruxy to anyone.
Maybe you have an additional claim like â...and growth faster than mooreâs law is unlikely?â If so, I would encourage you to write that because I think that is the kind of thing that would engage with peopleâs cruxes!
If you drew a chart for the GPT models on ARC-AGI-2, it would mostly just be a flat line.. Itâs only with the o3-low and o1-pro models we see scores above 0%
⌠which is what (super)-exponential growth looks like, yes?
Specifically: Weâve gone from o1 (low) getting 0.8% to o3 (low) getting 4% in ~1 year, which is ~2 doublings per year (i.e. 4x Mooreâs law). Forecasting from this few data points sure seems like a cursed endeavor to me, but if you want to do it then I donât see how you can rule out Mooreâs-law-or-faster growth.
I would be curious to know what the best benchmarks are which show a sub-Mooreâs-law trend.
Progress does not seem like a fast exponential trend, faster than Mooreâs law and laying the groundwork for an intelligence explosion
Mooreâs law is ~1 doubling every 2 years. Barnesâ law is ~4 doublings every 2 years:
This post is focused on what the government can do but Iâm curious if you have thoughts about what the private sector can do to meet the government where it is.
I imagine that palantir is making a killing off of adapting generative AI to work for government requirements, but I assume there are still gaps in the marketplace? Do you have a sense for what these gaps are? is there some large segment of the government which would use generative AI if only it was compliant with standard X?
the other hand though some leadership jobs might not be the right job fit if theyâre not up for that kind of critique
Yeah, this used to be my take but a few iterations of trying to hire for jobs which exclude shy awkward nerds from consideration when the EA candidate pool consists almost entirely of shy awkward nerds has made the cost of this approach quite salient to me.
There are trade-offs to everything đ¤ˇââď¸
Only the most elite 0.1 percent of people can even have a meaningful âpublic private disconnectâ as you have to have quite a prominent public profile for that to even be an issue.
Hmm yeah, thatâs kinda my point? Like complaining about your annoying coworker anonymously online is fine, but making a public blog post like âmy coworker Jane Doe sucks for these reasonsâ would be weird, people get fired for stuff like that. And referencing their wedding website would be even more extreme.
(Of course, most peopleâs coworkers arenât trying to reshape the lightcone without public consent so idk, maybe different standards should apply here. I can tell you that a non-trivial number of people Iâve wanted to hire for leadership positions in EA have declined for reasons like âI donât want people critiquing my personal life on the EA Forumâ though.)
fwiw I think in any circle Iâve been a part of critiquing someone publicly based on their wedding website would be considered weird/âa low blow. (Including corporate circles.) [1]
- ^
I think there is a level of influence at which everything becomes fair game, e.g. Donald Trump canât really expect a public/âprivate communication disconnect. I donât think thatâs true of Daniela, although I concede that her influence over the light cone might not actually be that much lower than Trumpâs.
- ^
Fixed the link. I also tried your original prompt and it worked for me.
But interesting! The âHarder word, much vaguer clueâ seems to prompt it to not actually play hangman and instead antagonistically try to post hoc create a word after each guess which makes your guess wrong. I asked âDid you come up with a word when you first told me the number of letters or are you changing it after each guess?â And it said âI picked the word up front when I told you it was 10 letters long, and I havenât changed it since. Youâre playing against that same secret word the whole time.â (Despite me being able to see its reasoning trace that this is not what itâs doing.) When I say I give up it says âIâm sorryâI actually lost track of the word Iâd originally picked and canât accurately reveal it now.â (Because it realized that there was no word consistent with its clues, as you noted.)
So I donât think itâs correct to say that it doesnât know how to play hangman. (It knows, as you noted yourself.) It just wants so badly to make you lose that it lies about the word.