Joe is a former Reliability Engineer who pivoted careers in an effort to make AI go well. He’s taught courses in AI governance at BlueDot Impact, and consulted with the Center for AI Risk Management and Alignment. He now works on the communications team at the Machine Intelligence Research Institute.
Joe Rogero
Karma: 309
A conversation on concentration of power
Announcing my retirement to a life of entirely failing to desperately seek renewed meaning
We do not live by course alone
I felt like most of the counterarguments that I see in the wild (e.g. from people on Twitter, who are mostly much more informed about AI than the audience of this book) were left unaddressed. I have no idea whether the authors’ prioritization of counterarguments was right for that audience, and I do think it would be handy to have a version of this book somewhat more appropriate for AI twitter people.
PSA: The online resources do indeed contain quite a few counter-counterarguments that didn’t fit into the book. (Buck probably knows this already, some readers might not.)
So You Want to Work at a Frontier AI Lab
What We Can Do to Prevent Extinction by AI
The primary benefit I’m imagining is a single well-placed whistleblower positioned to publicly sound the alarm on a particularly obvious and immediate threat, perhaps related to CBRN capabilities. A better answer requires a longer post, which is in the works but may take a while.
Honestly this writeup did update me somewhat in favor of at least a few competent safety-conscious people working at major labs, if only so the safety movement has some access to what’s going on inside the labs if/when secrecy grows. The marginal extra researcher going to Anthropic, though? Probably not.
E.g. Ajeya’s median estimate is 99% automation of fully-remote jobs in roughly 6-8 years, 5+ years earlier than her 2023 estimate.
This seems more extreme than the linked comment suggests? I can’t find anything in the comment justifying “99% automation of fully-remote jobs”.
Frankly I think we get ASI and everyone dies before we get anything like 99% automation of current remote jobs, due to bureaucratic inertia and slow adoption. Automation of AI research comes first on the jagged frontier. I don’t think Ajeya disagrees?
Cost, Not Sacrifice
Registrations Open for 2024 NYC Secular Solstice & Megameetup
It’s often in the nature of thought experiments to try to reduce complicated things to simple choices. In reality, humans rarely know enough to do an explicit EV calculation about a decision correctly. It can still be an ideal that can help guide our decisions, such that “this seems like a poor trade of EV” is a red flag the same way “oh, I notice I could be Dutch booked by this set of preferences” is a good way to notice there may be a flaw in our thinking somewhere.
Flipping Out: The Cosmic Coinflip Thought Experiment Is Bad Philosophy
Mentorship in AGI Safety: Applications for mentorship are open!
Mentorship in AGI Safety (MAGIS)
Virtual AI Safety Unconference 2024
Impact Colabs started something similar but then abandoned it. They have a forum post and more detailed write-up on why. Our aim is less ambitious (for now, just listing and ranking project ideas with some filtering options) though we do hope to expand the list to include more active volunteer management options eventually. Of note, this database is divided into “quick wins”—roughly, things someone could do with less than a week’s work without being part of a particular organization—and “larger projects”—which typically involve starting a full-time group or or supporting an existing one.
If you know of a project not listed, feel free to add it!
I’d like to discuss a similar “metaproject” I have in the works. Currently my goal for a “minimum viable product” is just the list, with volunteer matching added later if it works, but also including smaller “quick win” projects and immediate contributions that could be made. Would you be willing to share further and discuss lessons learned on this one?
Not sure if prewritten material counts, but I’d like to enter my Trial of the Automaton if it qualifies. I can transfer it to Google docs if need be.
I agree that doing things takes time. If someone does not have the slack in their lives to do anything other than scrape by, I don’t recommend they force themselves. (I do recommend they call a representative about stopping the AI race. That takes mere minutes.) It’s not healthy to try to shoulder the world’s burdens when one’s knees are already buckling. This post is for everyone else.
If any org I’ve worked for meaningfully “controlled the narrative”, the world would look very different than it does. The narrative, such as it is, does not look very controlled to me.
I have seen many good people make changes happen simply by doing good work on their own time. Does this require slack, runway, and no small amount of luck? Sure. Do good and competent people have less reach than a sane and functional society would afford them? Probably. But one does have to actually take shots on goal in order to score, even when most of them miss, and that’s no less true for sounding vaguely like something out of a self-help book.
If you truly believe that impressing some gatekeeping organization is necessary to doing good work, then by all means set out to impress them. Sometimes it’s indeed necessary; for instance, I don’t see an international halt to AI development arriving without someone getting the U.S. government on board.
But I’ve taken direction from a high school dropout. The credentials bar is lower than you think.