Hi, I run the 80,000 Hours job board, thanks for writing this out!
I agree that OpenAI has demonstrated a significant level of manipulativeness and have lost confidence in them prioritizing existential safety work. However, we donât conceptualize the board as endorsing organisations. The point of the board is to give job-seekers access to opportunities where they can contribute to solving our top problems or build career capital to do so (as we write in our FAQ). Sometimes these roles are at organisations whose mission I disagree with, because the role nonetheless seems like an opportunity to do good work on a key problem.
For OpenAI in particular, weâve tightened up our listings since the news stories a month ago, and are now only posting infosec roles and direct safety work â a small percentage of jobs they advertise. See here for the OAI roles we currently list. We used to list roles that seemed more tangentially safety-related, but because of our reduced confidence in OpenAI, we limited the listings further to only roles that are very directly on safety or security work. I still expect these roles to be good opportunities to do important work. Two live examples:
Even if we were very sure that OpenAI was reckless and did not care about existential safety, I would still expect them to not want their model to leak out to competitors, and importantly, we think itâs still good for the world if their models donât leak! So I would still expect people working on their infosec to be doing good work.
These still seem like potentially very strong roles with the opportunity to do very important work. We think itâs still good for the world if talented people work in roles like this!
This is true even if we expect them to lack political power and to play second fiddle to capabilities work and even if that makes them less good opportunities vs. other companies.
We also include a note on their âjob cardsâ on the job board (also DeepMindâs and Anthropicâs) linking to the Working at an AI company article you mentioned, to give context. Weâre not opposed to giving more or different context on OpenAIâs cards and are happy to take suggestions!
I wonât argue more for removing infosec roles at the moment. As noted in the post, I think this is at least a reasonable position to hold. I (weakly) disagree, but for reasons that donât seem worth getting into here.
The things Iâd argue here:
Safetywashing is actually pretty bad, for the worldâs epistemics and for EA and AI safetyâs collective epistemics. I think it also warps the epistemics of the people taking the job, so while they might be getting some career experience⊠theyâre also likely getting a distorted view of what what AI safety is, and becoming worse researchers than they would otherwise.
As previously stated â itâs not that I donât think anyone should take these jobs, but I think the sort of person who should take them is someone who has a higher degree of context and skill than I expect the 80k job board to filter for.
Even if you disagree with those points, you should have some kind of crux for âwhat would distinguish an âimpactful AI safety job?ââ vs a fake safety-washed role. It should be at least possible for OpenAI to make a role so clearly fake that you notice and stop listing it.
If youâre set on continuing to list OpenAI Alignment roles, I think the current disclaimer is really inadequate and misleading. (Partly because of the object-level content in Working at an AI company, which I think wrongly characterizes OpenAI, and partly because what disclaimers you do have are deep in that post. On the top-level job ad, thereâs no indication that applicants should be skeptical about OpenAI.
Re: cruxes for safetywashing
Youâd presumably agree, OpenAI couldnât just call any old job âAlignment Scienceâ and have it automatically count as worth listing on your site.
Companies at least sometimes lie, and they often use obfuscating language to mislead. OpenAIâs track record is such that, we know that they do lie and mislead. So, IMO, your prior here should be moderately high.
Maybe you only think itâs, like, 10%? (or less? IMO less than 10% feels pretty strained to me). But, at what credence would you stop listing it on the job board? And what evidence would increase your odds?
We are seeking Researchers to help design and implement experiments for alignment research. Responsibilities may include:
Writing performant and clean code for ML training
Independently running and analyzing ML experiments to diagnose problems and understand which changes are real improvements
Writing clean non-ML code, for example when building interfaces to let workers interact with our models or pipelines for managing human data
Collaborating closely with a small team to balance the need for flexibility and iteration speed in research with the need for stability and reliability in a complex long-lived project
Understanding our high-level research roadmap to help plan and prioritize future experiments
Designing novel approaches for using LLMs in alignment research
Want to use your engineering skills to push the frontiers of what state-of-the-art language models can accomplish
Possess a strong curiosity about aligning and understanding ML models, and are motivated to use your career to address this challenge
Enjoy fast-paced, collaborative, and cutting-edge research environments
Have experience implementing ML algorithms (e.g., PyTorch)
Can develop data visualization or data collection interfaces (e.g., JavaScript, Python)
Want to ensure that powerful AI systems stay under human control.
Is this an alignment research role, or a capabilities role that pays token lip service to alignment?My guess (60%) based on my knowledge of OpenAI is itâs more like the latter.
It says âDesigning novel approaches for using LLMs in alignment researchâ, but thatâs only useful if you think OpenAI uses the phrase âalignment researchâ to mean something important. We know they eventually coined the term âsuperalignmentâ, distinguished from most of what theyâd been calling âalignmentâ work (where âsuperalignmentâ is closer to what was originally meant by the term).
If OpenAI was creating jobs that werenât really helpful at all, but labeling them âalignmentâ anyway, how would you know?
Re: On whether OpenAI could make a role that feels insufficiently truly safety-focused: there have been and continue to be OpenAI safety-ish roles that we donât list because we lack confidence theyâre safety-focused.
For the alignment role in question, I think the team description given at the top of the post gives important context for the roleâs responsibilities:
OpenAIâs Alignment Science research teams are working on technical approaches to ensure that AI systems reliably follow human intent even as their capabilities scale beyond human ability to directly supervise them.
With the above in mind, the role responsibilities seem fine to me. I think this is all pretty tricky, but in general, Iâve been moving toward looking at this in terms of the teams:
Alignment Science: Per the above team description, Iâm excited for people to work there â though, concerning the question of what evidence would shift me, this would change if the research they release doesnât match the team description.
Preparedness: I continue to think itâs good for people to work on this team, as per the description: âThis team ⊠is tasked with identifying, tracking, and preparing for catastrophic risks related to frontier AI models.â
Safety Systems: I think roles here depend on what they address. I think the problems listed in their team description include problems I definitely want people working on (detecting unknown classes of harm, red-teaming to discover novel failure cases, sharing learning across industry, etc), but itâs possible that we should be more restrictive in which roles we list from this team.
I donât feel confident giving a probability here, but I do think thereâs a crux here around me not expecting the above team descriptions to be straightforward lies. Itâs possible that the teams will have limited resources to achieve their goals, and with the Safety Systems team in particular, I think thereâs an extra risk of safety work blending into product work. However, my impression is that the teams will continue to work on their stated goals.
I do think itâs worthwhile to think of some evidence that would shift me against listing roles from a team:
If a team doesnât publish relevant safety research within something like a year.
If a teamâs stated goal is updated to have less safety focus.
Other notes:
Weâre actually in the process of updating the AI company article.
The top-level disclaimer: Yeah, I think this needs updating to something more concrete. We put it up while âeverything was happeningâ but Iâve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
Thanks for diving into the implicit endorsement point. I acknowledge this could be a problem (and if so, I want to avoid it or at least mitigate it), so Iâm going to think about what to do here.
Fwiw while writing the above, I did also think âhmm, I should also have some cruxes for âwhat would update me towards âthese jobs are more real than I currently think.ââ Iâm mulling that over and will write up some thoughts soon.
It sounds like you basically trust their statements about their roles. I appreciate you stating your position clearly, but, I do think this position doesnât make sense:
we already have evidence of them failing to uphold commitments theyâve made in clear cut ways. (i.e. Iâd count their superalignment compute promises as basically a straightforward lie, and if not a âlieâ, it at least clearly demonstrates that their written words donât count for much. This seems straightforwardly relevant to the specific topic of âwhat does a given job at OpenAI entail?â, in addition to being evidence about their overall relationship with existential safety)
weâve similarly seen OpenAI change itâs stated policies, such as removing restrictions on military use. Or, initially being a nonprofit and converting into âfor-profit-managed by non-profitâ (where the âmanaged by nonprofit boardâ part turned out to be pretty ineffectual) (not sure if I endorse this, mulling over Habrykaâs comment)
Surely, this at at least updates you downward on how trustworthy their statements are? How many times do they have to âsay things that turned out not to be trueâ before you stop taking them at face value? And why is that âmore times than they have already?â.
Separate from straightforward lies, and/âor altering of policy to the point where any statements they make seem very unreliable, there is plenty of degrees of freedom of âwhat counts as alignment.â They are already defining alignment in a way that is pretty much synonymous with short-term capabilities. I think the plan of âiterate on âalignmentâ with nearterm systems as best you can to learn and prepareâ is not necessarily a crazy plan. There are people I respect who endorse it, who previously defended it as an OpenAI approach, although notably most of those people have now left OpenAI (sometimes still working on similar plans at other orgs).
But, itâs very hard to tell the difference from the outside between:
âiterating on nearterm systems, contributing to AI race dynamics in the process, in a way that has a decent chance of teaching you skills that will be relevant for aligning superintelligencesâ
âiterating on nearterm systems, in a way that you think/âhope will teach you skills for navigating superintelligence⊠but, youâre wrong about how much youâre learning, and whether itâs net positiveâ
âiterating on nearterm systems, and calling it alignment because it makes for better PR, but not even really believing that itâs particularly necessary to navigate superintelligence.
When recommending jobs for organizations that are potentially causing great harm, I think 80k has a responsibility to actually form good opinions on whether the job makes sense, independent on what the organization says itâs about.
You donât just need to model whether OpenAI is intentionally lying, you also need to model whether they are phrasing things ambiguously, and you need to model whether they are self-decelving about whether these roles are legitimate alignment work, or valuable enough work to outweigh the risks. And, you need to model that they might just be wrong and incompetent at longterm alignment development (or: âinsufficiently competent to outweigh risks and downsidesâ), even if their heart were in the right place.
I am very worried that this isnât already something you have explicit models about.
As Iâve discussed in the comments on a related post, I donât think OpenAI meaningfully changed any of its stated policies with regards to military usage. I donât think OpenAI really ever promised anyone they wouldnât work with militaries, and framing this as violating a past promise weakens the ability to hold them accountable for promises they actually made.
What OpenAI did was to allow more users to use their product. Itâs similar to LessWrong allowing crawlers or jurisdictions that we previously blocked to now access the site. I certainly wouldnât consider myself to have violated some promise by allowing crawlers or companies to access LessWrong that I had previously blocked (or for a closer analogy, letâs say we were currently blocking AI companies from crawling LW for training purposes, and I then change my mind and do allow them to do that, I would not consider myself to have broken any kind of promise or policy).
The arguments you give all sound like reasons OpenAI safety positions could be beneficial. But I find them completely swamped by all the evidence that they wonât be, especially given how much evidence OpenAI has hidden via NDAs.
But letâs assume weâre in a world where certain people could do meaningful safety work an OpenAI. What are the chances those people need 80k to tell them about it? OpenAI is the biggest, most publicized AI company in the world; if Alice only finds out about OpenAI jobs via 80k thatâs prima facie evidence she wonât make a contribution to safety.
What could the listing do? Maybe Bob has heard of OAI but is on the fence about applying. An 80k job posting might push him over the edge to applying or accepting. The main way I see that happening is via a halo effect from 80k. The mere existence of the posting implies that the job is aligned with EA/â80kâs values.
I donât think thereâs a way to remove that implication with any amount of disclaimers. The job is still on the board. If anything disclaimers make the best case scenarios seem even better, because why else would you host such a dangerous position?
So let me ask: what do you see as the upside to highlighting OAI safety jobs on the job board? Not of the job itself, but the posting. Who is it that would do good work in that role, and the 80k job board posting is instrumental in them entering it?
Update: Weâve changed the language in our top-level disclaimers: example. Thanks again for flagging! Weâre now thinking about how to best minimize the possibility of implying endorsement.
To try to be a bit more helpful rather than just complaining and arguing: when I model your current worldview, and try to imagine a disclaimer that helps a bit more with my concerns but seems like it might work for you given your current views, hereâs a stab. Changes bolded.
OpenAI is a frontier AI research and product company, with teams working on alignment, policy, and security. We recommend specific opportunities at OpenAI that we think may be high impact. We recommend applicants pay attention to the details of individual roles at OpenAI, and form their own judgment about whether the role is net positive. We do not necessarily recommend working at other positions at OpenAI
(itâs not my main crux, by âfrontierâ felt both like a more up-to-date term for what OpenAI does, and also feels more specifically like itâs making a claim about the product than generally awarding status to the company the way âleadingâ does)
I canât find the disclaimer. Not saying it isnât there. But it should be obvious from just skimming the page, since that is what most people will do.
I think that given the 80k brand (which is about helping people to have a positive impact with their career), itâs very hard for you to have a jobs board which isnât kinda taken by many readers as endorsement of the orgs. Disclaimers help a bit, but itâs hard for them to address the core issue â because for many of the orgs you list, you basically do endorse the org (AFAICT).
I also think itâs a pretty different experience for employees to turn up somewhere and think they can do good by engaging in a good faith way to help the org do whatever itâs doing, and for employees to not think that but think itâs a good job to take anyway.
My take is that you would therefore be better splitting your job board into two sections:
In one section, only include roles at orgs where you basically feel happy standing behind them, and think itâs straightforwardly good for people to go there and help the orgs be better
You can be conservative about inclusion here â and explain in the FAQ that non-inclusion in this list doesnât mean that it wouldnât be good to straightforwardly help the org, just that this isnât transparent enough to 80k to make the recommendation
In another more expansive section, you could list all roles which may be impactful, following your current strategy
If you only have this section (as at present), which I do think is a legit strategy, I feel you should probably rebrand it to something other than â80k job boardâ, and it should live somewhere other than the 80k website â even if you continue to link it prominently from the 80k site
Otherwise I think that you are in part spending 80kâs reputation in endorsing these organizations
If you have both of the sections Iâm suggesting, I still kind of think it might be better to have this section not under the 80k brand â but I feel that much less strongly, since the existence of the first section should help a good amount in making it transparent that listing doesnât mean endorsement of the org
(If the binary in/âout seems too harsh, you could potentially include gradations, like âFully endorsed orgâ, âPartially endorsed orgâ, and so on)
Edited to add: I want to acknowledge that at a gut level Iâm very sympathetic to âbut we explained all of this clearly!â. I do think if people misunderstand you then thatâs partially on them. But I also think that itâs sort of predictable that they will misunderstand in this way, and so itâs also partially on you.
I should also say that Iâm not confident in this read. If someone did a survey and found that say <10% of people browsing the 80k job board thought that 80k was kind of endorsing the orgs, Iâd be surprised but Iâd also update towards thinking that your current approach was a good one.
I am generally very wary of trying to treat your audience as unsophisticated this way. I think 80k taking on the job of recommending the most impactful jobs, according to the best of their judgement, using the full nuance and complexity of their models, is much clearer and straightforward than a recommendation which is something like âthe most impactful jobs, except when we donât like being associated with something, or where the case for it is a bit more complicated than our other jobs, or where our funders asked us to not include it, etc.â.
I do think that doing this well requires the ability to sometimes say harsh things about an organization. I think communicating accurately about job recommendations will inevitably require being able to say âwe think working at this organization might be really miserable and might involve substantial threats, adversarial relationships, and you might cause substantial harm if you are not careful, but we still think itâs overall still a good choice if you take that into accountâ. And I think those judgements need to be made on an organization-by-organization level (and canât easily be captured by generic statements in the context of the associated career guide).
I donât think you should treat your audience as unsophisticated. But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think itâs worth doing something to minimise the extent to which they come away misinformed.
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) ⊠I think itâs very reasonable to just skip this whole dynamic.
But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think itâs worth doing something to minimise the extent to which they come away misinformed.
Yeah, I agree with this. I like the idea of having different kinds of sections, and I am strongly in favor of making things be true at an intuitive glance as well as on closer reading (I like something in the vicinity of âThe Onion Testâ here)
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) ⊠I think itâs very reasonable to just skip this whole dynamic.
I feel like this dynamic is just fine? I definitely donât think you should recommend that they hide the fact they listened to you, that seems very deceptive. I think you tell people your honest opinion, and then if the other side retaliates, you take it. I definitely donât think 80k should send people to work at organizations as some kind of secret agent, and I think responding by protecting OpenAIs reputation by not disclosing crucial information about the role, feels like straightforwardly giving into an unjustified threat.
Hmm at some level Iâm vibing with everything youâre saying, but I still donât think I agree with your conclusion. Trying to figure out whatâs going on there.
Maybe itâs something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesnât mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
With these norms, if youâre blunt, you encourage people to read you as saying something worse than is true, or to read you as having an inability to act courteously. Neither of which are messages Iâd be keen to send.
And I sort of think these norms are good, because theyâre softly de-escalatory in terms of verbal spats or ill feeling. When people feel attacked itâs easy for them to be a little irrational and vilify the other side. If everyone is blunt publicly I think this can escalate minor spats into major fights.
Maybe itâs something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesnât mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
I donât really think these are the prevailing norms, especially not regards with an adversary who has leveraged illegal threats of destroying millions of dollars of value to prevent negative information from getting out.
Separately about whether these are the norms, I think the EA community plays a role in society where being honest and accurate about our takes of other people is important. There were a lot of people who took what the EA community said about SBF and FTX seriously and this caused enormous harm. In many ways the EA community (and 80k in-particular) are playing the role of a rating agency, and as a rating agency you need to be able to express negative ratings, otherwise you fail at your core competency.
As such, even if there are some norms in society about withholding negative information here, I think the EA and AI-safety communities in-particular cannot hold itself to these norms within the domains of their core competencies and responsibilities.
I donât regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing whatâs pertinent, or something?
Honestly Iâm not sure how much we really disagree here. I guess weâd have to concretely discuss wording for an org. In the case of OpenAI, I imagine it being appropriate to include some disclaimer like:
OpenAI is a frontier AI company. It has repeatedly expressed an interest in safety and has multiple safety teams. However, some people leaving the company have expressed concern that it is not on track to handle AGI safely, and that it wasnât giving its safety teams resources they had been promised. Moreover, it has a track record of putting inappropriate pressure on people leaving the company to sign non-disparagement agreements. [With links]
I donât regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing whatâs pertinent, or something?
I agree with some definitions of âfriendlyâ here, and disagree with others. I think there is an attractor here towards Orwellian language that is intentionally ambiguous about what itâs trying to say, in order to seem friendly or non-threatening (because in some sense it is), and that kind of âfriendlyâ seems pretty bad to me.
I think the paragraph you have would strike me as somewhat too Orwellian, though itâs not too far off from what I would say. Something closer to what seems appropriate to me:
OpenAI is a frontier AI company, and as such itâs responsible for substantial harm by assisting in the development of dangerous AI systems, which we consider among the biggest risks to humanityâs future. In contrast to most of the jobs in our job board, we consider working at OpenAI more similar to working at a large tobacco company, hoping to reduce the harm that the tobacco company causes, or leveraging this specific tobacco companyâs expertise with tobacco to produce more competetive and less harmful variations of tobacco products.
To its credit, it has repeatedly expressed an interest in safety and has multiple safety teams, which are attempting to reduce the likelihood of catastrophic outcomes from AI systems.
However, many people leaving the company have expressed concern that it is not on track to handle AGI safely, that it wasnât giving its safety teams resources they had been promised, and that the leadership of the company is untrustworthy. Moreover, it has a track record of putting inappropriate pressure on people leaving the company to sign non-disparagement agreements. [With links]
We explicitly recommend against taking any roles not in computer security or safety at OpenAI, and consider those substantially harmful under most circumstances (though exceptions might exist).
I feel like this is currently a bit too âedgyâ or something, and I would massage some sentences for longer, but it captures the more straightforward style that i think would be less likely to cause people to misunderstand the situation.
So it may be that we just have some different object-level views here. I donât think I could stand behind the first paragraph of what youâve written there. Hereâs a rewrite that would be palatable to me:
OpenAI is a frontier AI company, aiming to develop artificial general intelligence (AGI). We consider poor navigation of the development of AGI to be among the biggest risks to humanityâs future. It is complicated to know how best to respond to this. Many thoughtful people think it would be good to pause AI development; others think that it is good to accelerate progress in the US. We think both of these positions are probably mistaken, although we wouldnât be shocked to be wrong. Overall we think that if we were able to slow down across the board that would probably be good, and that steps to improve our understanding of the technology relative to absolute progress with the technology are probably good. In contrast to most of the jobs in our job board, therefore, it is not obviously good to help OpenAI with its mission. It may be more appropriate to consider working at OpenAI as more similar to working at a large tobacco company, hoping to reduce the harm that the tobacco company causes, or leveraging this specific tobacco companyâs expertise with tobacco to produce more competetive and less harmful variations of tobacco products.
I want to emphasise that this difference is mostly not driven by a desire to be politically acceptable (although the inclusion/âwording of the âmany thoughtful people âŠâ clauses are a bit for reasons of trying to be courteous), but rather a desire not to give bad advice, nor to be overconfident on things.
⊠That paragraph doesnât distinguish at all between OpenAI and, say, Anthropic. Surely you want to include some details specific to the OpenAI situation? (Or do your object-level views really not distinguish between them?)
I was just disagreeing with Habrykaâs first paragraph. Iâd definitely want to keep content along the lines of his third paragraph (which is pretty similar to what I initially drafted).
Otherwise I think that you are in part spending 80kâs reputation in endorsing these organizations
Agree on this. For a long time Iâve had a very low opinion of 80kâs epistemics[1] (both podcast, and website), and having orgs like OpenAI and Meta on there was a big contributing factor[2].
In particular that they try to both present as an authoritative source on strategic matters concerning job selection, while not doing the necessary homework to actually claim such status & using articles (and parts of articles) that empirically nobody reads & Iâve found are hard to find to add in those clarifications, if they ever do.
I think this is a good policy and broadly agree with your position.
Itâs a bit awkward to mention, but as youâve said that youâve delisted other roles at OpenAI and that OpenAI has acted badly beforeâI think you should consider explicitly saying that you donât necessarily endorse other roles at OpenAI and suspect that some other role may be harmful on the OpenAI jobs board cards.
Iâm a little worried about people seeing OpenAI listed on the board and inferring that the 80k recommendation somewhat transfers to other roles at OpenAI (which, imo is a reasonable heuristic for most companies listed on the boardâbut fails in this specific case).
Fwiw, I donât think that being on the 80k podcast is much of an endorsement of the work that people are doing. I think the signal is much more like âwe think this person is impressive and interestingâ, which is consistent with other âinterview podcastsâ (and I suspect that itâs especially true of podcasts that are popular amongst 80k listeners).
I also think having OpenAI employees discuss their views publicly with smart and altruistic people like Rob is generally pretty great, and I would personally be excited for 80k to have more OpenAI employees (particularly if they are willing to talk about why they do/âdonât think AIS is important and talk about their AI worldview).
Having a line at the start of the podcast making it clear that they donât necessarily endorse the org the guest works for would mitigate most concernsâthough I donât think itâs particularly necessary.
I would agree with this if 80k didnât make it so easy for the podcast episodes to become PR vehicles for the companies: some time back 80k changed their policy and now they send all questions to interviewees in advance, and let them remove any answers they didnât like upon reflection. Both of these make it very straightforward for the companiesâ PR teams to influence what gets said in an 80k podcast episode, and remove any confidence that youâre getting an accurate representation of the researcherâs views, rather than what the PR team has approved them to say.
These still seem like potentially very strong roles with the opportunity to do very important work. We think itâs still good for the world if talented people work in roles like this!
I think given that these jobs involved being pressured via extensive legal blackmail into signing secret non-disparagement agreements that forced people to never criticize OpenAI, at great psychological stress and at substantial cost to many outsiders who were trying to assess OpenAI, I donât agree with this assessment.
Safety people have been substantially harmed by working at OpenAI, and safety work at OpenAI can have substantial negative externalities.
Insofar as you are recommending the jobs but not endorsing the organization, I think it would be good to be fairly explicit about this in the job listing. The current short description of OpenAI seems pretty positive to me:
OpenAI is a leading AI research and product company, with teams working on alignment, policy, and security. You can read more about considerations around working at a leading AI company in our career review on the topic. They are also currently the subject of news stories relating to their safety work.
I think this should say something like âWe recommend jobs at OpenAI because we think these specific positions may be high impact. We would not necessarily recommend working at other jobs at OpenAI (especially jobs which increase AI capabilities).â
I also donât know what to make of the sentence âThey are also currently the subject of news stories relating to their safety work.â Is this an allusion to the recent exodus of many safety people from OpenAI? If so, I think itâs misleading and gives far too positive an impression.
Relatedly, I think that the âShould you work at a leading AI company?â article shouldnât start with a pros and cons list which sort of buries the fact that you might contribute to building extremely dangerous AI.
I think âRisk of contributing to the development of harmful AI systemsâ should at least be at the top of the cons list. But overall this sort of reminds me of my favorite graphic from 80k:
?? Itâs the second bullet point in the cons list, and reemphasized in the third bullet?
If youâre saying âobviously this is the key determinant of whether you should work at a leading AI company so there shouldnât even be a pros /â cons tableâ, then obviously 80K disagrees given they recommend some such roles (and many other people (e.g. me) also disagree so this isnât 80K ignoring expert consensus). In that case I think you should try to convince 80K on the object level rather than applying political pressure.
This thread feels like a fine place for people to express their opinion as a stakeholder.
Like, I donât even know how to engage with 80k staff on this on the object level, and seems like the first thing to do is to just express my opinion (and like, they can then choose to respond with argument).
Yeah, I think this needs updating to something more concrete. We put it up while âeverything was happeningâ but Iâve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
we donât conceptualize the board as endorsing organisations.
And
contribute to solving our top problems or build career capital to do so
It seems like EAs expect the 80k job board to suggest high impact roles, and this has been a misunderstanding for a long time (consider looking at that post if you havenât). The disclaimers were always there, but EAs (including myself) still regularly looked at the 80k job board as a concrete path to impact.
I donât have time for a long comment, just wanted to say I think this matters.
I donât read those two quotes as in tension? The job board isnât endorsing organizations, itâs endorsing roles. An organization can be highly net harmful while the right person joining to work on the right thing can be highly positive.
I also think âendorsementâ is a bit too strong: the bar for listing a job shouldnât be âanyone reading this who takes this job will have significant positive impactâ but instead more like âunder some combinations of values and world models that the job board runners think are plausible, this job is plausibly one of the highest impact opportunities for the right personâ.
My own intuition on what to do with this situationâis to stop trying to change your reputation using disclaimers.
Thereâs a lot of value in having a job board with high impact job recommendations. One of the challenging parts is getting a critical mass of people looking at your job board, and you already have that.
What are the relevant disclaimers here? Conor is saying 80l does think that alignment roles at OpenAI are impactful. Your article mentions the career development tag, but the roles under discussion donât have that tag right?
If Conor thinks these roles are impactful then Iâm happy we agree on listing impactful roles. (The discussion on whether alignment roles are impactful is separate from what I was trying to say in my comment)
If the career development tag is used (and is clear to typical people using the job board) thenâagainâseems good to me.
We used to list roles that seemed more tangentially safety-related, but because of our reduced confidence in OpenAI...
This misses aspects of what used to be 80kâs position:
â In fact, we think it can be the best career step for some of our readers to work in labs, even in non-safety roles. Thatâs the core reason why we list these roles on our job board. â Benjamin Hilton, February 2024
â Top AI labs are high-performing, rapidly growing organisations. In general, one of the best ways to gain career capital is to go and work with any high-performing team â you can just learn a huge amount about getting stuff done. They also have excellent reputations more widely. So you get the credential of saying youâve worked in a leading lab, and youâll also gain lots of dynamic, impressive connections. â Benjamin Hilton, June 2023 - still on website
80k was listing some non-safety related jobs: â From my email on May 2023:
Echoing Raemon, itâs still a value judgement about an organisation to say that 80k believes that a given role is one where, as you say, âthey can contribute to solving our top problems or build career capital to do soâ. You are saying that you have sufficient confidence that the organisation is run well enough that someone with little context of internal politics and pressures that canât be communicated via a job board can come in and do that job impactfully.
But such a person would be very surprised to learn that previous people in their role or similar ones at the company have not been able to do their job due to internal politics, lies, obfuscation etc, and that they may not be able to do even the basics of their job (see the broken promise of dedicated compute supply).
Itâs difficult to even build career capital as a technical researcher when youâre not given the resources to do your job and instead find yourself having to upskill in alliance building and interpersonal psychology.
I have slightly complex thoughts about the âis 80k endorsing OpenAI?â question.
Iâm generally on the side of âlet people make individual statements without treating it as a blanket endorsement.â
In practice, I think the job postings will be read as an endorsement by many (most?) people. But I think the overall policy of âsocial-pressure people to stop making statements that could be read as endorsementsâ is net harmful.
I think you should at least be acknowledging the implication-of-endorsement as a cost you are paying.
Iâm a bit confused about how to think about it here, because I do think listing people on the job site, with the sorts of phrasing you use, feels more like some sort of standard corporate political move than a purely epistemic move.
I do want to distinguish the question of âhow does this job-ad funnel social status around?â from âdoes this job-ad communicate clearly?â. I think itâs still bad to force people only speak words that canât be inaccurately read into, but, I think this is an important enough area to put extra effort in.
An accurate job posting, IMO, would say âOpenAI-in-particular has demonstrated that they do not follow through on safety promises, and weâve seen people leave due to not feeling effectual.â
I think you maybe both disagree with that object level fact (if so, I think you are wrong, and this is important), as well as, well, thatâd be a hell of a weird job ad. Part of why I am arguing here is I think it looks, from the outside, like 80k is playing a slightly confused mix of relating to orgs politically and making epistemic recommendations.
I kind of expect at this point you to leave the job ad up, and maybe change the disclaimer slightly in a way that is leaves some sort of plausibly-deniable veneer.
Hi, I run the 80,000 Hours job board, thanks for writing this out!
I agree that OpenAI has demonstrated a significant level of manipulativeness and have lost confidence in them prioritizing existential safety work. However, we donât conceptualize the board as endorsing organisations. The point of the board is to give job-seekers access to opportunities where they can contribute to solving our top problems or build career capital to do so (as we write in our FAQ). Sometimes these roles are at organisations whose mission I disagree with, because the role nonetheless seems like an opportunity to do good work on a key problem.
For OpenAI in particular, weâve tightened up our listings since the news stories a month ago, and are now only posting infosec roles and direct safety work â a small percentage of jobs they advertise. See here for the OAI roles we currently list. We used to list roles that seemed more tangentially safety-related, but because of our reduced confidence in OpenAI, we limited the listings further to only roles that are very directly on safety or security work. I still expect these roles to be good opportunities to do important work. Two live examples:
Infosec
Even if we were very sure that OpenAI was reckless and did not care about existential safety, I would still expect them to not want their model to leak out to competitors, and importantly, we think itâs still good for the world if their models donât leak! So I would still expect people working on their infosec to be doing good work.
Non-infosec safety work
These still seem like potentially very strong roles with the opportunity to do very important work. We think itâs still good for the world if talented people work in roles like this!
This is true even if we expect them to lack political power and to play second fiddle to capabilities work and even if that makes them less good opportunities vs. other companies.
We also include a note on their âjob cardsâ on the job board (also DeepMindâs and Anthropicâs) linking to the Working at an AI company article you mentioned, to give context. Weâre not opposed to giving more or different context on OpenAIâs cards and are happy to take suggestions!
Nod, thanks for the reply.
I wonât argue more for removing infosec roles at the moment. As noted in the post, I think this is at least a reasonable position to hold. I (weakly) disagree, but for reasons that donât seem worth getting into here.
The things Iâd argue here:
Safetywashing is actually pretty bad, for the worldâs epistemics and for EA and AI safetyâs collective epistemics. I think it also warps the epistemics of the people taking the job, so while they might be getting some career experience⊠theyâre also likely getting a distorted view of what what AI safety is, and becoming worse researchers than they would otherwise.
As previously stated â itâs not that I donât think anyone should take these jobs, but I think the sort of person who should take them is someone who has a higher degree of context and skill than I expect the 80k job board to filter for.
Even if you disagree with those points, you should have some kind of crux for âwhat would distinguish an âimpactful AI safety job?ââ vs a fake safety-washed role. It should be at least possible for OpenAI to make a role so clearly fake that you notice and stop listing it.
If youâre set on continuing to list OpenAI Alignment roles, I think the current disclaimer is really inadequate and misleading. (Partly because of the object-level content in Working at an AI company, which I think wrongly characterizes OpenAI, and partly because what disclaimers you do have are deep in that post. On the top-level job ad, thereâs no indication that applicants should be skeptical about OpenAI.
Re: cruxes for safetywashing
Youâd presumably agree, OpenAI couldnât just call any old job âAlignment Scienceâ and have it automatically count as worth listing on your site.
Companies at least sometimes lie, and they often use obfuscating language to mislead. OpenAIâs track record is such that, we know that they do lie and mislead. So, IMO, your prior here should be moderately high.
Maybe you only think itâs, like, 10%? (or less? IMO less than 10% feels pretty strained to me). But, at what credence would you stop listing it on the job board? And what evidence would increase your odds?
Taking the current Alignment Science researcher role as an example:
Is this an alignment research role, or a capabilities role that pays token lip service to alignment?My guess (60%) based on my knowledge of OpenAI is itâs more like the latter.
It says âDesigning novel approaches for using LLMs in alignment researchâ, but thatâs only useful if you think OpenAI uses the phrase âalignment researchâ to mean something important. We know they eventually coined the term âsuperalignmentâ, distinguished from most of what theyâd been calling âalignmentâ work (where âsuperalignmentâ is closer to what was originally meant by the term).
If OpenAI was creating jobs that werenât really helpful at all, but labeling them âalignmentâ anyway, how would you know?
Re: On whether OpenAI could make a role that feels insufficiently truly safety-focused: there have been and continue to be OpenAI safety-ish roles that we donât list because we lack confidence theyâre safety-focused.
For the alignment role in question, I think the team description given at the top of the post gives important context for the roleâs responsibilities:
OpenAIâs Alignment Science research teams are working on technical approaches to ensure that AI systems reliably follow human intent even as their capabilities scale beyond human ability to directly supervise them.
With the above in mind, the role responsibilities seem fine to me. I think this is all pretty tricky, but in general, Iâve been moving toward looking at this in terms of the teams:
Alignment Science: Per the above team description, Iâm excited for people to work there â though, concerning the question of what evidence would shift me, this would change if the research they release doesnât match the team description.
Preparedness: I continue to think itâs good for people to work on this team, as per the description: âThis team ⊠is tasked with identifying, tracking, and preparing for catastrophic risks related to frontier AI models.â
Safety Systems: I think roles here depend on what they address. I think the problems listed in their team description include problems I definitely want people working on (detecting unknown classes of harm, red-teaming to discover novel failure cases, sharing learning across industry, etc), but itâs possible that we should be more restrictive in which roles we list from this team.
I donât feel confident giving a probability here, but I do think thereâs a crux here around me not expecting the above team descriptions to be straightforward lies. Itâs possible that the teams will have limited resources to achieve their goals, and with the Safety Systems team in particular, I think thereâs an extra risk of safety work blending into product work. However, my impression is that the teams will continue to work on their stated goals.
I do think itâs worthwhile to think of some evidence that would shift me against listing roles from a team:
If a team doesnât publish relevant safety research within something like a year.
If a teamâs stated goal is updated to have less safety focus.
Other notes:
Weâre actually in the process of updating the AI company article.
The top-level disclaimer: Yeah, I think this needs updating to something more concrete. We put it up while âeverything was happeningâ but Iâve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
Thanks for diving into the implicit endorsement point. I acknowledge this could be a problem (and if so, I want to avoid it or at least mitigate it), so Iâm going to think about what to do here.
Thanks.
Fwiw while writing the above, I did also think âhmm, I should also have some cruxes for âwhat would update me towards âthese jobs are more real than I currently think.ââ Iâm mulling that over and will write up some thoughts soon.
It sounds like you basically trust their statements about their roles. I appreciate you stating your position clearly, but, I do think this position doesnât make sense:
we already have evidence of them failing to uphold commitments theyâve made in clear cut ways. (i.e. Iâd count their superalignment compute promises as basically a straightforward lie, and if not a âlieâ, it at least clearly demonstrates that their written words donât count for much. This seems straightforwardly relevant to the specific topic of âwhat does a given job at OpenAI entail?â, in addition to being evidence about their overall relationship with existential safety)
weâve similarly seen OpenAI change itâs stated policies, such asremoving restrictions on military use. Or, initially being a nonprofit and converting into âfor-profit-managed by non-profitâ (where the âmanaged by nonprofit boardâ part turned out to be pretty ineffectual)(not sure if I endorse this, mulling over Habrykaâs comment)Surely, this at at least updates you downward on how trustworthy their statements are? How many times do they have to âsay things that turned out not to be trueâ before you stop taking them at face value? And why is that âmore times than they have already?â.
Separate from straightforward lies, and/âor altering of policy to the point where any statements they make seem very unreliable, there is plenty of degrees of freedom of âwhat counts as alignment.â They are already defining alignment in a way that is pretty much synonymous with short-term capabilities. I think the plan of âiterate on âalignmentâ with nearterm systems as best you can to learn and prepareâ is not necessarily a crazy plan. There are people I respect who endorse it, who previously defended it as an OpenAI approach, although notably most of those people have now left OpenAI (sometimes still working on similar plans at other orgs).
But, itâs very hard to tell the difference from the outside between:
âiterating on nearterm systems, contributing to AI race dynamics in the process, in a way that has a decent chance of teaching you skills that will be relevant for aligning superintelligencesâ
âiterating on nearterm systems, in a way that you think/âhope will teach you skills for navigating superintelligence⊠but, youâre wrong about how much youâre learning, and whether itâs net positiveâ
âiterating on nearterm systems, and calling it alignment because it makes for better PR, but not even really believing that itâs particularly necessary to navigate superintelligence.
When recommending jobs for organizations that are potentially causing great harm, I think 80k has a responsibility to actually form good opinions on whether the job makes sense, independent on what the organization says itâs about.
You donât just need to model whether OpenAI is intentionally lying, you also need to model whether they are phrasing things ambiguously, and you need to model whether they are self-decelving about whether these roles are legitimate alignment work, or valuable enough work to outweigh the risks. And, you need to model that they might just be wrong and incompetent at longterm alignment development (or: âinsufficiently competent to outweigh risks and downsidesâ), even if their heart were in the right place.
I am very worried that this isnât already something you have explicit models about.
As Iâve discussed in the comments on a related post, I donât think OpenAI meaningfully changed any of its stated policies with regards to military usage. I donât think OpenAI really ever promised anyone they wouldnât work with militaries, and framing this as violating a past promise weakens the ability to hold them accountable for promises they actually made.
What OpenAI did was to allow more users to use their product. Itâs similar to LessWrong allowing crawlers or jurisdictions that we previously blocked to now access the site. I certainly wouldnât consider myself to have violated some promise by allowing crawlers or companies to access LessWrong that I had previously blocked (or for a closer analogy, letâs say we were currently blocking AI companies from crawling LW for training purposes, and I then change my mind and do allow them to do that, I would not consider myself to have broken any kind of promise or policy).
Mmm, nod. I will look into the actual history here more, but, sounds plausible. (edited the previous comment a bit for now)
The arguments you give all sound like reasons OpenAI safety positions could be beneficial. But I find them completely swamped by all the evidence that they wonât be, especially given how much evidence OpenAI has hidden via NDAs.
But letâs assume weâre in a world where certain people could do meaningful safety work an OpenAI. What are the chances those people need 80k to tell them about it? OpenAI is the biggest, most publicized AI company in the world; if Alice only finds out about OpenAI jobs via 80k thatâs prima facie evidence she wonât make a contribution to safety.
What could the listing do? Maybe Bob has heard of OAI but is on the fence about applying. An 80k job posting might push him over the edge to applying or accepting. The main way I see that happening is via a halo effect from 80k. The mere existence of the posting implies that the job is aligned with EA/â80kâs values.
I donât think thereâs a way to remove that implication with any amount of disclaimers. The job is still on the board. If anything disclaimers make the best case scenarios seem even better, because why else would you host such a dangerous position?
So let me ask: what do you see as the upside to highlighting OAI safety jobs on the job board? Not of the job itself, but the posting. Who is it that would do good work in that role, and the 80k job board posting is instrumental in them entering it?
Update: Weâve changed the language in our top-level disclaimers: example. Thanks again for flagging! Weâre now thinking about how to best minimize the possibility of implying endorsement.
Following up my other comment:
To try to be a bit more helpful rather than just complaining and arguing: when I model your current worldview, and try to imagine a disclaimer that helps a bit more with my concerns but seems like it might work for you given your current views, hereâs a stab. Changes bolded.
(itâs not my main crux, by âfrontierâ felt both like a more up-to-date term for what OpenAI does, and also feels more specifically like itâs making a claim about the product than generally awarding status to the company the way âleadingâ does)
Thanks. This still seems pretty insufficient to me, but, itâs at least an improvement and I appreciate you making some changes here.
I canât find the disclaimer. Not saying it isnât there. But it should be obvious from just skimming the page, since that is what most people will do.
I think that given the 80k brand (which is about helping people to have a positive impact with their career), itâs very hard for you to have a jobs board which isnât kinda taken by many readers as endorsement of the orgs. Disclaimers help a bit, but itâs hard for them to address the core issue â because for many of the orgs you list, you basically do endorse the org (AFAICT).
I also think itâs a pretty different experience for employees to turn up somewhere and think they can do good by engaging in a good faith way to help the org do whatever itâs doing, and for employees to not think that but think itâs a good job to take anyway.
My take is that you would therefore be better splitting your job board into two sections:
In one section, only include roles at orgs where you basically feel happy standing behind them, and think itâs straightforwardly good for people to go there and help the orgs be better
You can be conservative about inclusion here â and explain in the FAQ that non-inclusion in this list doesnât mean that it wouldnât be good to straightforwardly help the org, just that this isnât transparent enough to 80k to make the recommendation
In another more expansive section, you could list all roles which may be impactful, following your current strategy
If you only have this section (as at present), which I do think is a legit strategy, I feel you should probably rebrand it to something other than â80k job boardâ, and it should live somewhere other than the 80k website â even if you continue to link it prominently from the 80k site
Otherwise I think that you are in part spending 80kâs reputation in endorsing these organizations
If you have both of the sections Iâm suggesting, I still kind of think it might be better to have this section not under the 80k brand â but I feel that much less strongly, since the existence of the first section should help a good amount in making it transparent that listing doesnât mean endorsement of the org
(If the binary in/âout seems too harsh, you could potentially include gradations, like âFully endorsed orgâ, âPartially endorsed orgâ, and so on)
Edited to add: I want to acknowledge that at a gut level Iâm very sympathetic to âbut we explained all of this clearly!â. I do think if people misunderstand you then thatâs partially on them. But I also think that itâs sort of predictable that they will misunderstand in this way, and so itâs also partially on you.
I should also say that Iâm not confident in this read. If someone did a survey and found that say <10% of people browsing the 80k job board thought that 80k was kind of endorsing the orgs, Iâd be surprised but Iâd also update towards thinking that your current approach was a good one.
I am generally very wary of trying to treat your audience as unsophisticated this way. I think 80k taking on the job of recommending the most impactful jobs, according to the best of their judgement, using the full nuance and complexity of their models, is much clearer and straightforward than a recommendation which is something like âthe most impactful jobs, except when we donât like being associated with something, or where the case for it is a bit more complicated than our other jobs, or where our funders asked us to not include it, etc.â.
I do think that doing this well requires the ability to sometimes say harsh things about an organization. I think communicating accurately about job recommendations will inevitably require being able to say âwe think working at this organization might be really miserable and might involve substantial threats, adversarial relationships, and you might cause substantial harm if you are not careful, but we still think itâs overall still a good choice if you take that into accountâ. And I think those judgements need to be made on an organization-by-organization level (and canât easily be captured by generic statements in the context of the associated career guide).
I donât think you should treat your audience as unsophisticated. But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think itâs worth doing something to minimise the extent to which they come away misinformed.
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) ⊠I think itâs very reasonable to just skip this whole dynamic.
Yeah, I agree with this. I like the idea of having different kinds of sections, and I am strongly in favor of making things be true at an intuitive glance as well as on closer reading (I like something in the vicinity of âThe Onion Testâ here)
I feel like this dynamic is just fine? I definitely donât think you should recommend that they hide the fact they listened to you, that seems very deceptive. I think you tell people your honest opinion, and then if the other side retaliates, you take it. I definitely donât think 80k should send people to work at organizations as some kind of secret agent, and I think responding by protecting OpenAIs reputation by not disclosing crucial information about the role, feels like straightforwardly giving into an unjustified threat.
Hmm at some level Iâm vibing with everything youâre saying, but I still donât think I agree with your conclusion. Trying to figure out whatâs going on there.
Maybe itâs something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesnât mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
With these norms, if youâre blunt, you encourage people to read you as saying something worse than is true, or to read you as having an inability to act courteously. Neither of which are messages Iâd be keen to send.
And I sort of think these norms are good, because theyâre softly de-escalatory in terms of verbal spats or ill feeling. When people feel attacked itâs easy for them to be a little irrational and vilify the other side. If everyone is blunt publicly I think this can escalate minor spats into major fights.
I donât really think these are the prevailing norms, especially not regards with an adversary who has leveraged illegal threats of destroying millions of dollars of value to prevent negative information from getting out.
Separately about whether these are the norms, I think the EA community plays a role in society where being honest and accurate about our takes of other people is important. There were a lot of people who took what the EA community said about SBF and FTX seriously and this caused enormous harm. In many ways the EA community (and 80k in-particular) are playing the role of a rating agency, and as a rating agency you need to be able to express negative ratings, otherwise you fail at your core competency.
As such, even if there are some norms in society about withholding negative information here, I think the EA and AI-safety communities in-particular cannot hold itself to these norms within the domains of their core competencies and responsibilities.
I donât regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing whatâs pertinent, or something?
Honestly Iâm not sure how much we really disagree here. I guess weâd have to concretely discuss wording for an org. In the case of OpenAI, I imagine it being appropriate to include some disclaimer like:
I largely agree with the rating-agency frame.
I agree with some definitions of âfriendlyâ here, and disagree with others. I think there is an attractor here towards Orwellian language that is intentionally ambiguous about what itâs trying to say, in order to seem friendly or non-threatening (because in some sense it is), and that kind of âfriendlyâ seems pretty bad to me.
I think the paragraph you have would strike me as somewhat too Orwellian, though itâs not too far off from what I would say. Something closer to what seems appropriate to me:
I feel like this is currently a bit too âedgyâ or something, and I would massage some sentences for longer, but it captures the more straightforward style that i think would be less likely to cause people to misunderstand the situation.
So it may be that we just have some different object-level views here. I donât think I could stand behind the first paragraph of what youâve written there. Hereâs a rewrite that would be palatable to me:
I want to emphasise that this difference is mostly not driven by a desire to be politically acceptable (although the inclusion/âwording of the âmany thoughtful people âŠâ clauses are a bit for reasons of trying to be courteous), but rather a desire not to give bad advice, nor to be overconfident on things.
⊠That paragraph doesnât distinguish at all between OpenAI and, say, Anthropic. Surely you want to include some details specific to the OpenAI situation? (Or do your object-level views really not distinguish between them?)
I was just disagreeing with Habrykaâs first paragraph. Iâd definitely want to keep content along the lines of his third paragraph (which is pretty similar to what I initially drafted).
Yeah, this paragraph seems reasonable (I disagree, but like, thatâs fine, it seems like a defensible position).
Yeah same. (although, this focuses entirely on their harm as an AI organization, and not manipulative practices)
I think it leaves the question âwhat actually is the above-the-fold-summaryâ (whichâd be some kind of short tag).
Agree on this. For a long time Iâve had a very low opinion of 80kâs epistemics[1] (both podcast, and website), and having orgs like OpenAI and Meta on there was a big contributing factor[2].
In particular that they try to both present as an authoritative source on strategic matters concerning job selection, while not doing the necessary homework to actually claim such status & using articles (and parts of articles) that empirically nobody reads & Iâve found are hard to find to add in those clarifications, if they ever do.
Probably second to their horrendous SBF interview.
I think this is a good policy and broadly agree with your position.
Itâs a bit awkward to mention, but as youâve said that youâve delisted other roles at OpenAI and that OpenAI has acted badly beforeâI think you should consider explicitly saying that you donât necessarily endorse other roles at OpenAI and suspect that some other role may be harmful on the OpenAI jobs board cards.
Iâm a little worried about people seeing OpenAI listed on the board and inferring that the 80k recommendation somewhat transfers to other roles at OpenAI (which, imo is a reasonable heuristic for most companies listed on the boardâbut fails in this specific case).
I think this halo effect could be reduced by making small UI changes:
Removing the OpenAI logo
Replacing the âOpenAIâ name in the search results by âHarmful frontier AI Labâ or similar
Starting with a disclaimer on why this specific job might be good despite the overall org being bad
I would be all for a cleanup of 80k material to remove mentions of OpenAI as a place to improve the world.
The first two bullets donât seem like small UI changes to me; the second, in particular, seems too adversarial imo.
fwiw I donât think replacing the OpenAI logo or name makes much sense.
I do think itâs pretty important to actively communicate that even the safety roles shouldnât be taken at face value.
I agree with your second point Caleb, which is also why I think 80k need to stop having OpenAI (or similar) employees on their podcast.
Why? Because employer brand Halo Effects are real and significant.
Fwiw, I donât think that being on the 80k podcast is much of an endorsement of the work that people are doing. I think the signal is much more like âwe think this person is impressive and interestingâ, which is consistent with other âinterview podcastsâ (and I suspect that itâs especially true of podcasts that are popular amongst 80k listeners).
I also think having OpenAI employees discuss their views publicly with smart and altruistic people like Rob is generally pretty great, and I would personally be excited for 80k to have more OpenAI employees (particularly if they are willing to talk about why they do/âdonât think AIS is important and talk about their AI worldview).
Having a line at the start of the podcast making it clear that they donât necessarily endorse the org the guest works for would mitigate most concernsâthough I donât think itâs particularly necessary.
I would agree with this if 80k didnât make it so easy for the podcast episodes to become PR vehicles for the companies: some time back 80k changed their policy and now they send all questions to interviewees in advance, and let them remove any answers they didnât like upon reflection. Both of these make it very straightforward for the companiesâ PR teams to influence what gets said in an 80k podcast episode, and remove any confidence that youâre getting an accurate representation of the researcherâs views, rather than what the PR team has approved them to say.
I think given that these jobs involved being pressured via extensive legal blackmail into signing secret non-disparagement agreements that forced people to never criticize OpenAI, at great psychological stress and at substantial cost to many outsiders who were trying to assess OpenAI, I donât agree with this assessment.
Safety people have been substantially harmed by working at OpenAI, and safety work at OpenAI can have substantial negative externalities.
Insofar as you are recommending the jobs but not endorsing the organization, I think it would be good to be fairly explicit about this in the job listing. The current short description of OpenAI seems pretty positive to me:
I think this should say something like âWe recommend jobs at OpenAI because we think these specific positions may be high impact. We would not necessarily recommend working at other jobs at OpenAI (especially jobs which increase AI capabilities).â
I also donât know what to make of the sentence âThey are also currently the subject of news stories relating to their safety work.â Is this an allusion to the recent exodus of many safety people from OpenAI? If so, I think itâs misleading and gives far too positive an impression.
Relatedly, I think that the âShould you work at a leading AI company?â article shouldnât start with a pros and cons list which sort of buries the fact that you might contribute to building extremely dangerous AI.
I think âRisk of contributing to the development of harmful AI systemsâ should at least be at the top of the cons list. But overall this sort of reminds me of my favorite graphic from 80k:
?? Itâs the second bullet point in the cons list, and reemphasized in the third bullet?
If youâre saying âobviously this is the key determinant of whether you should work at a leading AI company so there shouldnât even be a pros /â cons tableâ, then obviously 80K disagrees given they recommend some such roles (and many other people (e.g. me) also disagree so this isnât 80K ignoring expert consensus). In that case I think you should try to convince 80K on the object level rather than applying political pressure.
This thread feels like a fine place for people to express their opinion as a stakeholder.
Like, I donât even know how to engage with 80k staff on this on the object level, and seems like the first thing to do is to just express my opinion (and like, they can then choose to respond with argument).
(Copied from reply to Raemon)
Yeah, I think this needs updating to something more concrete. We put it up while âeverything was happeningâ but Iâve neglected to change it, which is my mistake and will probably prioritize fixing over the next few days.
Hey Conor!
Regarding
And
It seems like EAs expect the 80k job board to suggest high impact roles, and this has been a misunderstanding for a long time (consider looking at that post if you havenât). The disclaimers were always there, but EAs (including myself) still regularly looked at the 80k job board as a concrete path to impact.
I donât have time for a long comment, just wanted to say I think this matters.
I donât read those two quotes as in tension? The job board isnât endorsing organizations, itâs endorsing roles. An organization can be highly net harmful while the right person joining to work on the right thing can be highly positive.
I also think âendorsementâ is a bit too strong: the bar for listing a job shouldnât be âanyone reading this who takes this job will have significant positive impactâ but instead more like âunder some combinations of values and world models that the job board runners think are plausible, this job is plausibly one of the highest impact opportunities for the right personâ.
My own intuition on what to do with this situationâis to stop trying to change your reputation using disclaimers.
Thereâs a lot of value in having a job board with high impact job recommendations. One of the challenging parts is getting a critical mass of people looking at your job board, and you already have that.
What are the relevant disclaimers here? Conor is saying 80l does think that alignment roles at OpenAI are impactful. Your article mentions the career development tag, but the roles under discussion donât have that tag right?
If Conor thinks these roles are impactful then Iâm happy we agree on listing impactful roles. (The discussion on whether alignment roles are impactful is separate from what I was trying to say in my comment)
If the career development tag is used (and is clear to typical people using the job board) thenâagainâseems good to me.
Iâm still confused about what the misunderstanding is
This misses aspects of what used to be 80kâs position:
â In fact, we think it can be the best career step for some of our readers to work in labs, even in non-safety roles. Thatâs the core reason why we list these roles on our job board.
â Benjamin Hilton, February 2024
â Top AI labs are high-performing, rapidly growing organisations. In general, one of the best ways to gain career capital is to go and work with any high-performing team â you can just learn a huge amount about getting stuff done. They also have excellent reputations more widely. So you get the credential of saying youâve worked in a leading lab, and youâll also gain lots of dynamic, impressive connections.
â Benjamin Hilton, June 2023 - still on website
80k was listing some non-safety related jobs:
â From my email on May 2023:
â From my comment on February 2024:
Where do they say the handpicked line?
Could you quote which line you mean? Then I can mention where you can find it back
Itâs the last line in your second screenshot
They mentioned that line at the top of the 80k Job board.
Still do I see.
âHandpicked to help you tackle the worldâs most pressing problems with your career.â
https://ââjobs.80000hours.org/ââ
Echoing Raemon, itâs still a value judgement about an organisation to say that 80k believes that a given role is one where, as you say, âthey can contribute to solving our top problems or build career capital to do soâ. You are saying that you have sufficient confidence that the organisation is run well enough that someone with little context of internal politics and pressures that canât be communicated via a job board can come in and do that job impactfully.
But such a person would be very surprised to learn that previous people in their role or similar ones at the company have not been able to do their job due to internal politics, lies, obfuscation etc, and that they may not be able to do even the basics of their job (see the broken promise of dedicated compute supply).
Itâs difficult to even build career capital as a technical researcher when youâre not given the resources to do your job and instead find yourself having to upskill in alliance building and interpersonal psychology.
I have slightly complex thoughts about the âis 80k endorsing OpenAI?â question.
Iâm generally on the side of âlet people make individual statements without treating it as a blanket endorsement.â
In practice, I think the job postings will be read as an endorsement by many (most?) people. But I think the overall policy of âsocial-pressure people to stop making statements that could be read as endorsementsâ is net harmful.
I think you should at least be acknowledging the implication-of-endorsement as a cost you are paying.
Iâm a bit confused about how to think about it here, because I do think listing people on the job site, with the sorts of phrasing you use, feels more like some sort of standard corporate political move than a purely epistemic move.
I do want to distinguish the question of âhow does this job-ad funnel social status around?â from âdoes this job-ad communicate clearly?â. I think itâs still bad to force people only speak words that canât be inaccurately read into, but, I think this is an important enough area to put extra effort in.
An accurate job posting, IMO, would say âOpenAI-in-particular has demonstrated that they do not follow through on safety promises, and weâve seen people leave due to not feeling effectual.â
I think you maybe both disagree with that object level fact (if so, I think you are wrong, and this is important), as well as, well, thatâd be a hell of a weird job ad. Part of why I am arguing here is I think it looks, from the outside, like 80k is playing a slightly confused mix of relating to orgs politically and making epistemic recommendations.
I kind of expect at this point you to leave the job ad up, and maybe change the disclaimer slightly in a way that is leaves some sort of plausibly-deniable veneer.