Feel free to message me on here.
JackM
Karma: 4,044
The person who gets the role is obviously going to be highly intelligent, probably socially adept, and highly-qualified with experience working in AI etc. etc. OpenAI wouldn’t hire someone who wasn’t.
The question is do you want this person also to care about safety. If so I would think advertising on the EA job board would increase the chance of this.
If you think EAs or people who look at the 80K Hours job board are for some reason less good epistemically than others then you will have to explain why because I believe the opposite.
You’re referring to job boards generally but we’re talking about the 80K job board which is no typical job board.
I would expect someone who will do a good job to be someone going in wanting to stop OpenAI destroying the world. That seems to be someone who would read the 80K Hours job board. 80K is all about preserving the future.
They of course also have to be good at navigating organizational social scenes while holding onto their own epistemics which in my opinion are skills commonly found in the EA community!
I think the evidence we have from OpenAI is that it isn’t very helpful to “be a safety conscious person there
It’s insanely hard to have an outsized impact in this world. Of course it’s hard to change things from inside OpenAI, but that doesn’t mean we shouldn’t try. If we succeed it could mean everything. You’re probably going to have lower expected value pretty much anywhere else IMO, even if it does seem intractable to change things at OpenAI.
I think it’s especially not helpful if you’re a low-context person, who reads an OpenAI job board posting, and isn’t going in with a specific plan to operate in an adversarial environment.
Surely this isn’t the typical EA though?
If OpenAI doesn’t hire an EA they will just hire someone else. I’m not sure if you tackle this point directly (sorry if I missed it) but doesn’t it straightforwardly seem better to have someone safety-conscious in these roles rather than someone who isn’t safety-conscious?
To reiterate, it’s not like if we remove these roles from the job board that they will less likely be filled. They would still definitely be filled, just by someone less safety-conscious in expectation. And I’m not sure the person who would get the role would be “less talented” in expectation because there are just so many talented ML researchers—so I’m not sure removing roles from the job board would slow down capabilities development much if at all.
I get a sense that your argument is somewhat grounded in deontology/virtue ethics (i.e. “don’t support a bad organization”) but perhaps not so much in terms of consequentialism?
That’s great of course. I still wouldn’t have chosen your title. But thank you for spreading the word to those who have influence!
Thanks for sharing. Your post title is very misleading though. I wouldn’t be surprised if Mr Beast has never even heard of EA. I’m not against clickbaity titles which are more or less accurate but exaggerated, but “Mr Beast is now officially an EA!” seems simply incorrect. Not a huge deal, but I was quite excited when I clicked on this post only to be left a bit disappointed. May be worth clarifying in the text that Mr Beast hasn’t actually signalled agreement with EA principles.
Hmm, I don’t see why ensuring the best people go to Anthropic necessarily means they will take safety less seriously. I can actually imagine the opposite effect as if Anthropic catches up or even overtakes OpenAI then their incentive to cut corners should actually decrease because it’s more likely that they can win the race without cutting corners. Right now their only hope to win the race is to cut corners.
Ultimately what matters most is what the leadership’s views are. I suspect that Sam Altman never really cared that much about safety, but my sense is that the Amodeis do.
What are you suggesting? That if we direct safety conscious people to Anthropic that it will make it more likely that Anthropic will start to cut corners? Not sure what your point is.
I’ve just thought of a counter-argument to my point. If OpenAI isn’t safe it may be worth trying to ensure a safer AI lab (say Anthropic) wins the race to AGI. So it might be worth suggesting that talented people go to Anthropic rather than AGI, even if it is part of product or capabilities teams.
From an EA-perspective—yes, maybe.
But also it’s a personal decision. If you’re burnt out and fed up or you can’t bear to support an organization you disagree with then you may be better off quitting.
Also, quitting in protest can be a way to convince an organization to change course. It’s not always effective, but it’s certainly a strong message to leadership that you disapprove of what they’re doing which may at the very least get them thinking.
Even if OpenAI has gone somewhat off the rails, should we want more or fewer safety-conscious people at OpenAI? I would imagine more.
Mini GPI Paper Summaries (Sept 2023 - March 2024)
That is fair. I still think the idea that aligned superintelligent AI in the wrong hands can be very bad may be under-appreciated. The implication is that something like moral circle expansion seems very important at the moment to help mitigate these risks. And of course work to ensure that countries with better values win the race to powerful AI.
Well I’m assigning extinction a value of zero and a neutral world is any world that has some individuals but also has a value of zero. For example it could be a world where half of the people live bad (negative) lives and the other half live equivalently good (positive) lives. So the sum total of wellbeing adds up to zero.
A dystopia is one which is significantly negative overall. For example a world in which there are trillions of factory farmed animals that live very bad lives. A world with no individuals is a world without all this suffering.
Could it be more important to improve human values than to make sure AI is aligned?
Consider the following (which is almost definitely oversimplified):
ALIGNED AI
MISALIGNED AI
HUMANITY GOOD VALUES
UTOPIA
EXTINCTION
HUMANITY NEUTRAL VALUES
NEUTRAL WORLD
EXTINCTION
HUMANITY BAD VALUES
DYSTOPIA
EXTINCTION
For clarity, let’s assume dystopia is worse than extinction. This could be a scenario where factory farming expands to an incredibly large scale with the aid of AI, or a bad AI-powered regime takes over the world. Let’s assume neutral world is equivalent to extinction.
The above shows that aligning AI can be good, bad, or neutral. The value of alignment exactly depends on humanity’s values. Improving humanity’s values however is always good.
The only clear case where aligning AI beats improving humanity’s values is if there isn’t scope to improve our values further. An ambiguous case is whenever humanity has positive values in which case both improving values and aligning AI are good options and it isn’t immediately clear to me which wins.
The key takeaway here is that improving values is robustly good whereas aligning AI isn’t—alignment is bad if we have negative values. I would guess that we currently have pretty bad values given how we treat non-human animals and alignment is therefore arguably undesirable. In this simple model, improving values would become the overwhelmingly important mission. Or perhaps ensuring that powerful AI doesn’t end up in the hands of bad actors becomes overwhelmingly important (again, rather than alignment).
This analysis doesn’t consider the moral value of AI itself. It also assumed that misaligned AI necessarily leads to extinction which may not be accurate (perhaps it can also lead to dystopian outcomes?).
I doubt this is a novel argument, but what do y’all think?
I was surprised by your “dearest” and “mirror” tests.
Call the first the “dearest test.” When you have some big call to make, sit down with a person very dear to you—a parent, partner, child, or friend—and look them in the eyes. Say that you’re making a decision that will affect the lives of many people, to the point that some strangers might be hurt. Say that you believe that the lives of these strangers are just as valuable as anyone else’s. Then tell your dearest, “I believe in my decisions, enough that I’d still make them even if one of the people who could be hurt was you.”
Or you can do the “mirror test.” Look into the mirror and describe what you’re doing that will affect the lives of other people. See whether you can tell yourself, with conviction, that you’re willing to be one of the people who is hurt or dies because of what you’re now deciding. Be accountable, at least, to yourself.
These tests seem to rule out many actions that seem desirable to undertake. For example:
Creating and distributing a COVID vaccine: there is some small risk of serious side effects and it is likely that a small number of people will die even though many, many more will be saved. So this may not pass the “dearest” and “mirror” tests. Should we not create and distribute vaccines?
Launching a military operation to stop genocide: A leader may need to order military action to halt an ongoing genocide, knowing that some innocent civilians and their own soldiers will likely die even though many more will be saved. This may not pass the “dearest” and “mirror” tests. Should we just allow the genocide?
Do you bite the bullet here? Or accept that these tests may be flawed?
We can think of this stance as analogous to:
The utilitarian parent: “I care primarily about doing what’s best for humanity at large, but I wouldn’t want to neglect my children to such a strong degree that all defensible notions of how to be a decent parent state that I fucked up.”
I wonder if we don’t mind people privileging their own children because:
People love their kids too damn much and it just doesn’t seem realistic for people to neglect their children to help others.
A world in which it is normalised to neglect your children to “focus on humanity” is probably a bad world by utilitarian lights. A world full of child neglect just doesn’t seem like it would produce productive individuals who can make the world great. So even on an impartial view we wouldn’t want to promote child neglect.
Neither of these points are relevant in the case of privileging existing-and-sure-to-exist people/beings vs possible people/beings:
We don’t have some intense biologically-driven urge to help present people. For example, most people don’t seem to care all that much that a lot of present people are dying from malaria. So focusing on helping possible people/beings seems at least feasible.
We can’t use the argument that it is better from an impartial view to focus on existing-and-sure-to-exist people/beings because of the classic ‘future could be super-long’ argument.
And when you say that a person with totalist/strong longtermist life goals also chooses between two separate values (what their totalist axiology says versus existing people), I’m not entirely sure that’s true. Again, massive neglect of existing people just doesn’t seem like it would work out well for the long term—existing people are the ones that can make the future great! So even pure strong longtermists will want some decent investment into present people.
Looking forward to reading this. A quick note: in 3. Tomi’s argument that creating happy people is good your introductory text doesn’t match what is in the table.
Not sure I entirely agree with the second paragraph. The white paper outlines how philanthropy in this area is quite neglected, and there are organisations like LaMP which could certainly use more funding. Page 5 of the white paper also outlines bottlenecks in the process—even if firms do have strong incentives to acquire talent there can be informational gaps that prevent them from finding the best individuals, and similar informational gaps exist for the individuals that prevent them from actively utilising the best pathways.
Having said that I’m not claiming this is the best use of EA dollars—just posting for people’s information.
The link with the comprehensive list of organizations and projects that engage in depolarization doesn’t work. Here is the link: Organizations Bridging Divides | Resources | The Morton Deutsch International Center for Cooperation and Conflict Resolution | Teachers College, Columbia University
And the pdf: Organizations-Bridging-Divides-Nov-2020.pdf (columbia.edu)