lightcone maybe at lightcone
jacobjacob
Karma: 895
I think some were false. For example, I don’t get the stuff about mini-drones undermining nuclear deterrence, as size will constrain your batteries enough that you won’t be able to do much of anything useful. Maybe I’m missing something (modulo nanotech).
I think it’s very plausible scaling holds up, it’s plausible AGI becomes a natsec matter, it’s plausible it will affect nuclear deterrence (via other means), for example.
What do you disagree with?
I agree with much of Leopold’s empirical claims, timelines, and analysis. I’m acting on it myself in my planning as something like a mainline scenario.
Nonetheless, the piece exhibited some patterns that gave me a pretty strong allergic reaction. It made or implied claims like:
* a small circle of the smartest people believe this
* i will give you a view into this small elite group who are the only who are situationally aware
* the inner circle longed tsmc way before you
* if you believe me; you can get 100x richer—there’s still alpha, you can still be early
* This geopolitical outcome is “inevitable” (sic!)
* in the future the coolest and most elite group will work on The Project. “see you in the desert” (sic)
* Etc.Combined with a lot of retweets, with praise, on launch day, that were clearly coordinated behind the scenes; it gives me the feeling of being deliberately written to meme a narrative into existence via self-fulfilling prophecy; rather than inferring a forecast via analysis.
As a sidenote, this felt to me like an indication of how different the AI safety adjacent community is now to when I joined it about a decade ago. In the early days of this space, I expect a piece like this would have been something like “epistemically cancelled”: fairly strongly decried as violating important norms around reasoning and cooperation. I actually expect that had someone written this publicly in 2016, they would’ve plausibly been uninvited as a speaker to any EAGs in 2017.
I don’t particularly want to debate whether these epistemic boundaries were correct—I’d just like to claim that, empirically, I think they de facto would have been enforced. Though, if others who have been around have a different impression of how this would’ve played out, I’d be curious to hear.
It’s indeed helpful that Shakeel expressed those views, because now it’s clear where he’s at, and it will make it easier to relate to him as a journalist in future.
Happened to come across this old comment thread discussion whether holding too much Facebook stock was too risky. In the four years since the comment on Sep 21, 2020, Meta stock is up >100% and at an all time high. However, before reaching that point, it also had as large as a 60% drawdown vs the Sep 21 value, which occurred in late 2022 (notably, around the time of the FTX collapse).
This is heartbreaking. I’m so sorry.
(I haven’t read the full comment here and don’t want to express opinions about all its claims. But for people who saw my comments on the other post, I want to state for the record that based on what I’ve seen of Richard Hanania’s writing online, I think Manifest next year would be better without him. It’s not my choice, but if I organised it, I wouldn’t invite him. I don’t think of him as a “friend of EA”.)
No, I think this is again importantly wrong.
First, this was published in the Guardian US, not the Guardian.
The Guardian US does not have half the traffic of the NYTimes. It has about 15% the traffic, far as I can tell (source). The GuardianUS has 200k Twitter followers; The Guardian has 10M Twitter followers (so 2% of the following).
Second, I scrolled through all the tweets in the link you sent showing “praise”. I see the following:
Emile Torres with 250 likes.
Timnit Gebru’s new research org retweeting, 27 likes
A professor I don’t know supporting it, 117 likes
Shakeel being “glad to see the press picking it up”, 14 likes
A confusing amount of posts, maybe 10+, which retweet and get 0 likes and no engagement, and 10 that get 1-10 likes
Original tweet by the author of the article, 500 likes
Another journalist praising, 60 likes
You can of course compare this to:
Tweet from a usually EA-critical account with 161 likes, “This is just bad assignment work for whoever wrote this beat.”
Theo Jaffe critical tweet, 144 likes
Robin Hanson with 400 likes, complaining about defamation.
Byrne Hobart critical tweet, 500 likes
Multiple Kelsey tweets, with 300 likes
Habryka’s refutation, 450 likes
Quilette editor critical tweet, 100 likes
So I think this just clearly proves my point: the majorty of engagement of this article on Twitter is just commenting on it being a terrible hit piece.
The tiny wave of praise came mostly from folks well known for bad faith attacks on EA, a strange trickle of no-to-low engagement retweets, 1-2 genuine professors, and, well, Shakeel.
Ah! I was wrong to claim you made “no” such comments. I’ve edited my above comment.
Now, I of course notice how you only mention “lots of mistakes” after Jeffrey objects, and after it’s become clear that there is a big outpouring of hit piece criticism, and only little support.
Why were you glad about it before then?
Did you:
...not think it was a hit piece? (I think you’re a smart guy, and even a journalist yourself, so I’m kind of incredulous about you not picking up on the patterns here)
...or were you okay with the-amount-of-hit-piece-you-thought-it-was? (this is of course what I’m worried about, and why I am pursuing this so vigorously. I think this article crossed several very important epistemic red lines, and I will fight for those lines to remain intact, and will be very vocal about confronting journalists close to the community who don’t seem to respect them)
...or something else? (reality might of course be more complicated than my neatly packaged options above, so do feel free to explain)
In the follow-up tweet you say: “Glad to see the press picking [this story] up (though wish they made the rationalist/EA distinction clearer!)”
So far as I’ve found, you’ve made no comments indicating that you disagree with the problematic methodology of the piece, and two comments saying you were “delighted” and “glad” with parts of it. I think my quote is representative. I’ve updated my comment for clarity.
Nonetheless: how would you prefer to be quoted?
EDIT: Shakeel posted a comment pointing to a tweet of his “mistakes” in the post, and I was wrong to claim there were no comments.
It also just occurred to me that Shakeel’s first tweet about the article was I think(?) the first time it appeared on Twitter. It was actually made before the author’s themselves retweeted it. And also before any of the “hit piece” pushback had appeared.
Man yesterday this was at +20 karma and no it’s at −20. There seems to be a massive diurnal effect in how the votes on the forum swing.
I think both of those karma values are kind of extreme, and so find myself flipping my vote around. But wish I could leave an anchor vote like “if the vote diverges from value X, change my vote to point it back toward X”
I think Shakeel’s cited definition with my clarification here seems good; https://forum.effectivealtruism.org/posts/MHenxzydsNgRzSMHY/my-experience-at-the-controversial-manifest-2024?commentId=rB6pq5guAWcsAJrWx
I think the really key thing here is the bait-and-switch at play.
Insofar as “controversial” means “heated discussion of subject x”, let’s call that “x-controversial”.
Now the article generates heated discussion because of “being a hit piece”, and so is “hit-piece-controversial”. However, there’s then also heated discussion of racism on the forum, call that “racism-controversial”.
If we then unpack the argument made by Garrison above, it reads as “It is fair and accurate to label Manifest racism-controversial, because of a piece of reporting that was hit-piece controversial”—clearly an invalid argument.
Moreover, I don’t know that the forum discourse was necessarily that heated; and seems like there could be a good faith conversation here about an important topic (for example the original author has been super helpful in engaging with replies, I think). So it also seems lots of “heat” got imported from a different controversy.
Crucially, I think part of the adversarial epistemic playbook of this article, the journalist behind it, as well as your own Tweets and comments supporting it, is playing on ambiguities like this (bundling a bunch of different x-controversial and y-controversial things into one label “controversial”), and then using those as the basis to make sweeping accusations that “organisations [...] cut all ties with Manifold/Lightcone”.
That is what I’m objecting so strongly against.
React to this comment to convey opinions on:
“I think the argument for controversy from the Guardian article is invalid; but I do think Manifest should be labeled controversial for other arguments that I think are valid”
To clarify:
Claiming that Manifest is controversial because of the Guardian reporting—I’ll argue against this pretty strongly
Claiming that Manifest is controversial because of an independent set of good faith accounts from EA forum members—more legit and I can see the case (though I personally disagree)
Very importantly, Garrison’s comment was arguing using 1, not 2.
To perhaps help clarify the discourse, I’ll leave a comment below where people can react to signal “I think the argument for controversy from the Guardian article is invalid; but I do think Manifest should be labeled controversial for other arguments that I think are valid”
“My experience at Manifest 2024”
“My experience with controversial speakers at Manifest 2024”
“My perception of HBD discourse at and around Manifest 2024”
No. You pointing a finger and yelling “controversial!” doesn’t make something controversial any more than you yelling “racist” at people makes them racist.
I think controversial is a totally fair and accurate description of the event given that it was the subject of a very critical story from a major newspaper, which then generated lots of heated commentary online.
No, this argument is importantly invalid.
It was not a “critical story”. It was a hit piece engineered to cause reputational damage. This distinction really matters. (For people who wanted more receipts than my above comment about the adversarial intent, the journalist behind the article now also has sent a cryptic message eerily similar to a death threat(!!) in response to discussion of the article, by what appears to a political rival of theirs. This is not neutral reporting)
The majority of commentary I saw was complaining about the piece being a hit piece. See [1], [2], [3], [4], [5], [6] … . The piece was also community noted on twitter.
The event series LessOnline, Summer Camp, Manifest in total had 500+ guests, 500+ sessions, 70+ invited speakers, across a 10 day stretch. It was a large festival with a ton of different content.
I strongly reject the norm whereby a belligerent writer at a small news outlet can pick out a small slice of a large event, write an adversarial hit piece on it, have people complain about the piece’s journalistic integrity, get some activity as a result; and then have people claim the whole event could be “fairly and accurately” described as controversial(!)
The term is not fair and it is not accurate. Manifest was not controversial; I reject the label. Closest I think is right is “Manifest invited some controversial speakers”. Like this new article from today, for example, which says “the venue’s owners played host to a conference with controversial attendees”. That seems right, and that I encourage a conversation about!
You might want to make your point by appealing to the conference itself, but appealing to the guardian article and its effects really is not a valid argument. For the epistemic health of the community, I think it would be wise to stay way clear of the process that generated that term here.
(instead of making all comments on both places, ill continue discussing over at lesswrong https://www.lesswrong.com/posts/i5pccofToYepythEw/against-aschenbrenner-how-situational-awareness-constructs-a#Hview8GCnX7w4XSre )