Substack shill @ parhelia.substack.com
Conor Barnes š¶
Karma: 376
One example I can think of with regards to people āgraduatingā from philosophies is the idea that people can graduate out of arguably āadolescentā political philosophies like libertarianism and socialism. Often this looks like people realizing society is messy and that simple political philosophies donāt do a good job of capturing and addressing this.
However, I think EA as a philosophy is more robust than the above: There are opportunities to address the immense suffering in the world and to address existential risk, some of these opportunities are much more impactful than others, and itās worth looking for and then executing on these opportunities. I expect this to be true for a very long time.
In general I think effective giving is the best opportunity for most people. We often get fixated on the status of directly working on urgent problems, which I think is a huge mistake. Effective giving is a way to have a profound impact, and I donāt like to think of it as something just āfor mere mortalsāāI think thereās something really amazing about people giving a portion of their income every year to save lives and health, and I think doing so makes you as much an EA as somebody whose job itself is impactful.
Hi there, Iād like to share some updates from the last month.
Text during last update (July 5)
OpenAI is a leading AI research and product company, with teams working on alignment, policy, and security. We recommend specific positions at OpenAI that we think may be high impact. We do not necessarily recommend working at other jobs at OpenAI. You can read more about considerations around working at a leading AI company in our career review on the topic.
Text as of today:
OpenAI is a frontier AI research and product company, with teams working on alignment, policy, and security. We post specific opportunities at OpenAI that we think may be high impact. We do not necessarily recommend working at other positions at OpenAI. You can read concerns about doing harm by working at a frontier AI company in our career review on the topic. Note that there have also been concerns around OpenAIās HR practices.
The thinking behind these updates has been:We continue to get negative updates concerning OpenAI, so itās good for us to update our guidance accordingly.
While itās unclear exactly whatās going on with the NDAs (are they cancelled or are they not?), itās pretty clear that itās in the interest of users to know thereās something they should look into with regard to HR practices.
Weāve tweaked the language to āconcerns about doing harmā instead of āconsiderationsā for all three frontier labs to indicate more strongly that these are potentially negative considerations to make before applying.
We donāt go into much detail for the sake of length /ā people not glazing over themāmy guess is that the current text is the right length to have people notice it and then look into it more with our newly updated AI company article and the Washington Post link.
This is thanks to discussions within 80k and thanks to some of the comments here. While I suspect, @Raemon, that we still donāt align on important things, I nonetheless appreciate the prompt to think this through more and I believe that it has led to improvements!
I interpreted the title to mean āIs it a good idea to take an unpaid UN internship?ā, and it took a bit to realize that isnāt the point of the post. You might want to change the title to be clear about what part of the unpaid UN internship is the questionable part!
Update: Weāve changed the language in our top-level disclaimers: example. Thanks again for flagging! Weāre now thinking about how to best minimize the possibility of implying endorsement.
(Copied from reply to Raemon)
Yeah, I think this needs updating to something more concrete. We put it up while āeverything was happeningā but Iāve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
Re: On whether OpenAI could make a role that feels insufficiently truly safety-focused: there have been and continue to be OpenAI safety-ish roles that we donāt list because we lack confidence theyāre safety-focused.
For the alignment role in question, I think the team description given at the top of the post gives important context for the roleās responsibilities:
OpenAIās Alignment Science research teams are working on technical approaches to ensure that AI systems reliably follow human intent even as their capabilities scale beyond human ability to directly supervise them.
With the above in mind, the role responsibilities seem fine to me. I think this is all pretty tricky, but in general, Iāve been moving toward looking at this in terms of the teams:
Alignment Science: Per the above team description, Iām excited for people to work there ā though, concerning the question of what evidence would shift me, this would change if the research they release doesnāt match the team description.
Preparedness: I continue to think itās good for people to work on this team, as per the description: āThis team ā¦ is tasked with identifying, tracking, and preparing for catastrophic risks related to frontier AI models.ā
Safety Systems: I think roles here depend on what they address. I think the problems listed in their team description include problems I definitely want people working on (detecting unknown classes of harm, red-teaming to discover novel failure cases, sharing learning across industry, etc), but itās possible that we should be more restrictive in which roles we list from this team.
I donāt feel confident giving a probability here, but I do think thereās a crux here around me not expecting the above team descriptions to be straightforward lies. Itās possible that the teams will have limited resources to achieve their goals, and with the Safety Systems team in particular, I think thereās an extra risk of safety work blending into product work. However, my impression is that the teams will continue to work on their stated goals.
I do think itās worthwhile to think of some evidence that would shift me against listing roles from a team:
If a team doesnāt publish relevant safety research within something like a year.
If a teamās stated goal is updated to have less safety focus.
Other notes:
Weāre actually in the process of updating the AI company article.
The top-level disclaimer: Yeah, I think this needs updating to something more concrete. We put it up while āeverything was happeningā but Iāve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
Thanks for diving into the implicit endorsement point. I acknowledge this could be a problem (and if so, I want to avoid it or at least mitigate it), so Iām going to think about what to do here.
Hi, I run the 80,000 Hours job board, thanks for writing this out!
I agree that OpenAI has demonstrated a significant level of manipulativeness and have lost confidence in them prioritizing existential safety work. However, we donāt conceptualize the board as endorsing organisations. The point of the board is to give job-seekers access to opportunities where they can contribute to solving our top problems or build career capital to do so (as we write in our FAQ). Sometimes these roles are at organisations whose mission I disagree with, because the role nonetheless seems like an opportunity to do good work on a key problem.
For OpenAI in particular, weāve tightened up our listings since the news stories a month ago, and are now only posting infosec roles and direct safety work ā a small percentage of jobs they advertise. See here for the OAI roles we currently list. We used to list roles that seemed more tangentially safety-related, but because of our reduced confidence in OpenAI, we limited the listings further to only roles that are very directly on safety or security work. I still expect these roles to be good opportunities to do important work. Two live examples:
Even if we were very sure that OpenAI was reckless and did not care about existential safety, I would still expect them to not want their model to leak out to competitors, and importantly, we think itās still good for the world if their models donāt leak! So I would still expect people working on their infosec to be doing good work.
These still seem like potentially very strong roles with the opportunity to do very important work. We think itās still good for the world if talented people work in roles like this!
This is true even if we expect them to lack political power and to play second fiddle to capabilities work and even if that makes them less good opportunities vs. other companies.
We also include a note on their ājob cardsā on the job board (also DeepMindās and Anthropicās) linking to the Working at an AI company article you mentioned, to give context. Weāre not opposed to giving more or different context on OpenAIās cards and are happy to take suggestions!
I find the Leeroy Jenkins scenario quite plausible, though in this world itās still important to build the capacity to respond well to public support.
Hi Remmelt,
Just following up on this ā I agree with Benjaminās message above, but I want to add that we actually did add links to the āworking at an AI labā article in the org descriptions for leading AI companies after we published that article last June.
It turns out that a few weeks ago the links to these got accidentally removed when making some related changes in Airtable, and we didnāt notice these were missing ā thanks for bringing this to our attention. Weāve added these back in and think they give good context for job board users, and weāre certainly happy for more people to read our articles.
We also decided to remove the prompt engineer /ā librarian role from the job board, since we concluded itās not above the current bar for inclusion. I donāt expect everyone will always agree with the judgement calls we make about these decisions, but we take them seriously, and we think itās important for people to think critically about their career choices.
I think this is a joke, but for those who have less-explicit feelings in this direction:
I strongly encourage you to not join a totalizing community. Totalizing communities are often quite harmful to members and being in one makes it hard to reason well. Insofar as an EA org is a hardcore totalizing community, it is doing something wrong.
I really appreciated reading this, thank you.
Rereading your post, Iād also strongly recommend prioritizing finding ways to not spend all free time on it. Not only do I think that that level of fixating is one of the worst things people can do to make themselves suffer, it also makes it very hard to think straight and figure things out!
One thing Iāve seen suggested is dedicating time each day to use as research time on your questions. This is a compromise to free up the rest of your time to things that donāt hurt your head. And hang out with friends who are good at distracting you!
Iām really sorry youāre experiencing this. I think itās something more and more people are contending with, so you arenāt alone, and Iām glad you wrote this. As somebody whoās had bouts of existential dread myself, there are a few things Iād like to suggest:
With AI, we fundamentally do not know what is to come. Weāre all making our best guessesāas you can tell by finding 30 different diagnoses! This is probably a hint that we are deeply confused, and that we should not be too confident that we are doomed (or, to be fair, too confident that we are safe).
For this reason, it can be useful to practice thinking through the models on your own. Start making your own guesses! I also often find the technical and philosophical details beyond meābut that doesnāt mean we canāt think through the broad strokes. āHow confident am I that instrumental convergence is real?ā āDo I think evals for new models will become legally mandated?ā āDo I think they will be effective at detecting deception?ā At the least, this might help focus your content consumption instead of being an amorphous blob of dreadāI refer to it this way because I found the invasion of Ukraine sent me similarly reading as much as I could. Developing a model by focusing on specific, concrete questions (e.g. What events would presage a nuclear strike?) helped me transform my anxiety from āEverything about this worries meā into something closer to āEvents X and Y are probably bad, but event Z is probably goodā.
I find it very empowering to work on the problems that worry me, even though my work is quite indirect. AI safety labs have content writing positions on occasion. I work on the 80,000 Hours job board and we list roles in AI safety. Though these are often research and engineering jobs, itās worth keeping an eye out. Itās possible that proximity to the problem would accentuate your stress, to be fair, but I do think it trades against the feeling of helplessness!
C. S. Lewis has a take on dealing with the dread of nuclear extinction that Iām very fond of and think is applicable: āHow are we to live in an atomic age?ā I am tempted to reply: āWhy, as you would have lived in the sixteenth century when the plague visited London almost every year...ā
I hope this helps!
I hadnāt seen the previous dashboard, but I think the new one is excellent!
Thanks for the Possible Worlds Tree shout-out!
I havenāt had capacity to improve it (and wonāt for a long time), but I agree that a dashboard would be excellent. I think it could be quite valuable even if the number choice isnāt perfect.
HalĀiĀfax Monthly Meetup: AI Safety Discussion
āGive a man money for a boat, he already knows how to fishā would play off of the original formation!
Quite happy to see this on the forum!
Become conversational in Spanish so I can talk to my fianceĆ©ā²s family easily.
Work out ten times per month (3x/āweek with leeway)
Submit 12 short stories about transformative AI to publishers this year.
More details here. Ongoing mission: get a literary agent for my novel!