Yes, I agree. I think what we need to spend our effort on is convincing people that AI development is dangerous and needs to be handled very cautiously if at all, not that superintelligence is imminent and there’s NO TIME. I don’t think the exact level of urgency or the exact level of risk matters much after like p(doom)=5. The thing we need to convince people of is how to handle the risk.
A lot of AI Safety messages expect the audience to fill in most of the interpretive details—“As you can see, this forecast is very well-researched. ASI is coming. You take it from here.”—when actually what they need to know is what those claims mean for them and what they can do.
Holly Elmore ⏸️ 🔸
Karma: 6,909
I thought you were taking issue with the claim they were overdiscussed and asking where.
The areas where timelines are overdiscussed are numerous. Policy and technical safety career advice are the biggest ime.
The community at large – and certainly specific EAs trying to distance themselves now – couldn’t have done anything to prevent FTX. They think they could have, and they think others see them as responsible, but this is only because EA was the center of their universe.
Ding! ding! ding!You’re saying it nicely. I think it was irrationality and cowardice. I felt traumatized by the hysterical reaction and the abandoning of our shared work and community. I also felt angry about how my friends helped enemies of EA to destroy EA’s reputation for their own gain.
I’ve stopped identifying with EA as much bc PauseAI is big tent and doesn’t hold all the EA precepts (that and much of the community is hostile to advocacy and way too corrupted by the AI industry…), but I always explain the connection and say that I endorse EA principles when I’m asked about it. It’s important to me to defend the values!
Yes, PauseAI also takes place in time and anticipates events. I stand corrected.
I wasn’t arguing against thinking about the future at all, if that wasn’t clear. I’m against an obsession with calling when AI will hit a certain power level or when we’re cooked instead of picking a robust intervention and just doing it.
No, I do not expect the people who replace them (or them not being replaced) to have much of an effect. I do not think they are really helping and I don’t think their absence would really hurt. The companies are following their own agenda and they’ll do that with or without specifc people in those roles.
AI Safety Twitter, this Forum, Bay Area parties…
(I don’t particularly endorse any timeline, btw, partly bc I don’t think it’s a decision-relevant question for me.)
Much better epistemics and/or coordination—out of reach now, put potentially obtainable with stronger tech.
Why are these the same category and why are you writing coordination off as impossible? It’s not. We have literally done global nonproliferation treaties before.
This bizarre notion got embedded early in EA that technological feats are possible and solving coordination problems is impossible. It’s actually the opposite—alignment is not tractable and coordination is.
I think almost nothing would change at the labs, but that the EA AI Safety movement would become less impotent, more clear, and stand more of a chance of doing good.
I think it is hazardous bc it in some way ties their “success” to yours.
Yeah I tried to exempt AI 2027 from my critique. They are doing a lot more, and well.
I disagree that “we know what we need to know”. To me, the proper conversation about timelines isn’t just “when AGI”, but rather, “at what times will a number of things happen”, including various stages of post-AGI technology, and AI’s dynamics with the world as a whole. It incorporates questions like “what kinds of AIs will be present”.
See I think forecasts like that don’t really give us useful enough information about how to plan for future contingencies. I think we are deluded if we think we can make important moves based, for example, on the kinds of AIs that we project could be present in the future. The actual state of our knowledge is very coarse and we need to act accordingly. I really think the only prospective chance for impact is to do things that slow development and create real human and democratic oversight, and we have almost no chance of nudging the trajectory of development in a technical direction that works for us from here. (Maybe we will after we’ve secured the time and will to do so!)
Probably our crux is that I think the way society sees AI development morally is what matters here to navigate the straits, and the science is not going to be able to do the job in time. I care about developing a field of technical AI Safety but not if it comes at the expense of moral clarity that continuing to train bigger and bigger models is not okay before we know it will be safe. I would much rather rally the public to that message than try to get in the weak safety paper discourse game (which tbc I consider toothless and assume is not guiding Google’s strategy).
I think there was a time when it seemed like a good idea, back when the companies were small and there was more of a chance of setting their standards and culture. Back in 2016 I thought on balance we should try to put Safety people in OpenAI, for instance. OpenAI was supposed to be explicitly Safety-oriented, but any company’s safety division seemed like it might pay off to stock with Safety people.
I think everything had clearly changed around the chatGPT moment. The companies had a successful paradigm for making the models, the product was extremely valuable, and the race was very clearly on. At this time, EAs still believed that OpenAI and Anthropic were on their side because they had Safety teams (including many EAs) and talked a lot about Safety, in fact claiming to be developing AGI for the sake of Safety. Actual influence from EA employees to do things that were safe that weren’t good for the mission of those companies was already lost at this point, imo.
It was proven in the ensuing two years that the Safety teams at OpenAI were expendable. Sam Altman has used up and thrown away EA, and he no longer feels any need to pretend OpenAI cares about Safety, despite having very fluently talked to the talk for years before. He was happy to use the EA board members and the entire movement as scapegoats.
Anthropic is showing signs of going the same way. They do Safety research, but nothing stops them developing further, including former promises not to advance the frontier. The main thing they do is develop bigger and bigger models. They want to be attractive to natsec, and whether the actual decisionmakers at the top ultimately believe their agenda is for the sake of Safety or not, it’s clearly not up to the marginal Safety hire or hingeing on their research results. Other AI companies don’t even claim to care about Safety particularly.
So, I do not think it is effective to work at these places. But the real harm is that working for AI labs keeps EAs from speaking out about AI danger, whether because they are under NDA, or because they want to be hireable by a lab, or they want to cooperate with people working at labs, or because they defer to their friends and general social environment and so they think the labs are good (at least Anthropic). imo this price is unacceptably high, and EAs would have a lot more of the impact they hoped to get from being “in the room” at labs by speaking out and contributing to real external pressure and regulation.
Can you list what you see as the costs?
I will answer comments that ask sincerely for explanations of my worldview on this. I am aware there is a lot of evidence listing and dot-connecting I didn’t do here.
When dealing with an effectiveness-focused movement, our adversaries are further incentivised to prevent concrete results. While other movements will have to be destroyed through pressure, an effectiveness-focused movement will easily go away if you just prove to them that they can be more effective elsewhere.
100. It’s handing control of your actions to anyone who can play your language game.
I think, for the reason you describe, it is most effective to commit to your campaigns. You do the critical thinking first, but eventually you have to actually do the plan and give it a chance to have realized impact. The fairweather people who want every object-level action they take to be defensible as the most effective possible thing (imo the dominant EA take atm) are the ones who are wrong about effectiveness—they can’t execute a real world, multi-step plan for impact.
Constantly switching paths and always taking the next step that looks most effective, including to your critics and enemies, is a way to maximize option value, not a way to accomplish any multi-step real world plan.
Convincing such people that Anthropic is doing corpspeak and not just being perfectly reasonable or justified by 3D chess (with ultimate EA goals) would be a lot of progress...
It’s a huge problem in EA that people don’t take CoI that seriously as something that affect their thinking. They think they can solve every problem explicitly intellectually so corruption by money won’t happen to them.
Are you saying bc it’s not “surprising” it should be allowed? This rhetorical move of shaming your opponent for not having already gotten used to and therefore tolerating someone doing bad things I always find bizarre.
Honestly, I wasn’t thinking of you! People planning their individual careers is one of the better reasons to engage with timelines imo. It’s more the selection of interventions where I think the conversation is moot, not where and how individuals can connect to those interventions.
The hypothetical example of people abandoning projects that culminate in 2029 was actually inspired by PauseAI—there is a contingent of people who think protesting and irl organizing takes too long and that we should just be trying to go viral on social media. I think the irl protests and community is what make PauseAI a real force and we have greater impact, including by drawing social media attention, all along that path—not just once our protests are big.
That said, I do see a lot of people making the mistakes I mentioned about their career paths. I’ve had a number of people looking for career advice through PauseAI say things like, “well, obviously getting a PhD is ruled out”, as if there is nothing they can do to have impact until they have the PhD. I think being a PhD student can be a great source of authority and a flexible job (with at least some income, often) where you have time to organize a willing population of students! (That’s what I did with EA at Harvard.) The mistake here isn’t even really a timelines issue; it’s not modeling the impact distribution along a career path well. Seems like you’ve been covering this:
>I also agree many people should be on paths that build their leverage into the 2030s, even if there’s a chance it’s ‘too late’. It’s possible to get ~10x more leverage by investing in career capital / org building / movement building, and that can easily offset. I’ll try to get this message across in the new 80k AI guide.