Announcing PauseCon, the PauseAI conference.
Three days of workshops, panels, and discussions, culminating in our biggest protest to date.
Twitter: https://x.com/PauseAI/status/1915773746725474581
Apply now: https://pausecon.org
Joseph Miller
Karma: 715
PauseCon London 2025
<negativity>
This is cool, but the 10% Pledge is for life. If you’re primarily motivated by current events you may find it difficult to stick to your pledge 10 years from now.
</negativity>This is really great! Giving to charity is awesome and it’s especially impactful right now!
The next international PauseAI protest is taking place in one week in London, New York, Stockholm (Sunday 9th Feb), Paris (Mon 10 Feb) and many other cities around the world.
We are calling for AI Safety to be the focus of the upcoming Paris AI Action Summit. If you’re on the fence, take a look at Why I’m doing PauseAI.
Great point! Off the cuff I don’t think this massively changes considerations for PauseAI, but I’ll need to think about this.
Yep, maybe. I’m responding specifically to the vibe that this particular pre-print should make us more scared about AI.
As a technical person: AI is scary but this paper in particular is a nothing-burger. See my other comments.
No, there is no interesting new method here, it’s using LLM scaffolding to copy some files and run a script. It can only duplicate itself within the machine it has been given access to.
In order for AI to spread like a virus it would have to have some way to access new sources of compute, for which it would need be able to get money or the ability to hack into other servers. Neither of which current LLMs appear to be capable of.
Successful self-replication under no human assistance is the essential step for
AI to outsmart the human beingsThis seems clearly false. Replication (under their operationalization) is just another programming task that is not especially difficult. There’s no clear link between this task and self improvement, which would be a much harder ML task requiring very different types of knowledge and actions.
However, I do separately think we have passed the level of capabilities where it is responsible to keep improving AIs.
I’m confused why the comments aren’t more about cause prioritization as that’s the primary choice here. Maybe that’s too big of a discussion for this comment section.
This just seems like another annoying spam / marketing email. I basically never want any unnecessary emails from any company ever.
EA co-working spaces are the most impactful EA infrastructure that I’m aware of. And they are mostly underfunded.
This is particularly relevant given the recent letter from Anthropic on SB-1047.
I would like to see a steelman of the letter since it appears to me to significantly undermine Anthropic’s entire raison d’etre (which I understood to be: “have a seat at the table by being one of the big players—use this power to advocate for safer AI policies”). And I haven’t yet heard anyone in the AI Safety community defending it.
Anthropic are pushing for two key changes
not to be accountable for “pre-harm” enforcement of AI Safety standards (ie. wait for a catastrophe before enforcing any liability).
“if a catastrophic event does occur … the quality of the company’s SSP should be a factor in determining whether the developer exercised ‘reasonable care.’”. (ie. if your safety protocols look good, you can be let off the hook for the consequences of catastrophe).
Also significantly weakening whistleblower protections.
Ok thanks, I didn’t know that.
Nit: Beff Jezos was doxxed and repeating him name seems uncool, even if you don’t like him.
proximity [...] is obviously not morally important
People often claim that you have a greater obligation to those in your own country than to foreigners. I’m doubtful of this
imagining drowning children that there are a bunch of nearby assholes ignoring the child as he drowns. Does that eliminate your reason to save the child? No, obviously not
Your argument seems to be roughly an appeal to the intuition that moral principles should be simple—consistent across space and time, without weird edge cases, not specific to the circumstances of the event. But why should they be?Imo this is the mistake that people make when they haven’t internalized reductionism and naturalism. In other words they are moral realist or otherwise confused. When you realize that “morality” is just “preferences” with a bunch of pointless religious, mystical and philosophical baggage, the situation becomes clearer.
Because preferences are properties of human brains, not physical laws there is no particular reason to expect them to have low Kolmogorov complexity. And to say that you “should” actually be consistent about moral principles is an empty assertion that entirely rests on a hazy and unnatural definition of “should”.
Nonetheless, the piece exhibited some patterns that gave me a pretty strong allergic reaction. It made or implied claims like:
* a small circle of the smartest people believe this
* i will give you a view into this small elite group who are the only who are situationally aware
* the inner circle longed tsmc way before you
* if you believe me; you can get 100x richer—there’s still alpha, you can still be early
* This geopolitical outcome is “inevitable” (sic!)
* in the future the coolest and most elite group will work on The Project. “see you in the desert” (sic)
* Etc.These are not just vibes—they are all empirical claims (except the last maybe). If you think they are wrong, you should say so and explain why. It’s not epistemically poor to say these things if they’re actually true.
I also claim that I understand ethics.
“Good”, “bad”, “right”, “wrong”, etc. are words that people project their confusions about preferences / guilt / religion on to. They do not have commonly agreed upon definitions. When you define the words precisely the questions become scientific, not philosophical.
People are looking for some way to capture their intuitions that God above is casting judgement about the true value of things—without invoking supernatural ideas. But they cannot, because nothing in the world actually captures the spirit of this intuition (the closest thing is preferences). So they relapse into confusion, instead of accepting the obvious conclusion that moral beliefs are in the same ontological category as opinions (like “my favorite color is red”), not facts (like “the sky appears blue”).
I expect much of this will be largely subjective and have no objective fact of the matter, but it can be better informed by both empirical and philsophical research.
So I would say it is all subjective. But I agree that understanding algorithms will help us choose which actions satisfy our preferences. (But not that searching for explanations of the magic of conscious will help us decide which actions are good.)
See ai-plans.com