I don’t think you should treat your audience as unsophisticated. But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think it’s worth doing something to minimise the extent to which they come away misinformed.
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) … I think it’s very reasonable to just skip this whole dynamic.
But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think it’s worth doing something to minimise the extent to which they come away misinformed.
Yeah, I agree with this. I like the idea of having different kinds of sections, and I am strongly in favor of making things be true at an intuitive glance as well as on closer reading (I like something in the vicinity of “The Onion Test” here)
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) … I think it’s very reasonable to just skip this whole dynamic.
I feel like this dynamic is just fine? I definitely don’t think you should recommend that they hide the fact they listened to you, that seems very deceptive. I think you tell people your honest opinion, and then if the other side retaliates, you take it. I definitely don’t think 80k should send people to work at organizations as some kind of secret agent, and I think responding by protecting OpenAIs reputation by not disclosing crucial information about the role, feels like straightforwardly giving into an unjustified threat.
Hmm at some level I’m vibing with everything you’re saying, but I still don’t think I agree with your conclusion. Trying to figure out what’s going on there.
Maybe it’s something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesn’t mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
With these norms, if you’re blunt, you encourage people to read you as saying something worse than is true, or to read you as having an inability to act courteously. Neither of which are messages I’d be keen to send.
And I sort of think these norms are good, because they’re softly de-escalatory in terms of verbal spats or ill feeling. When people feel attacked it’s easy for them to be a little irrational and vilify the other side. If everyone is blunt publicly I think this can escalate minor spats into major fights.
Maybe it’s something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesn’t mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
I don’t really think these are the prevailing norms, especially not regards with an adversary who has leveraged illegal threats of destroying millions of dollars of value to prevent negative information from getting out.
Separately about whether these are the norms, I think the EA community plays a role in society where being honest and accurate about our takes of other people is important. There were a lot of people who took what the EA community said about SBF and FTX seriously and this caused enormous harm. In many ways the EA community (and 80k in-particular) are playing the role of a rating agency, and as a rating agency you need to be able to express negative ratings, otherwise you fail at your core competency.
As such, even if there are some norms in society about withholding negative information here, I think the EA and AI-safety communities in-particular cannot hold itself to these norms within the domains of their core competencies and responsibilities.
I don’t regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing what’s pertinent, or something?
Honestly I’m not sure how much we really disagree here. I guess we’d have to concretely discuss wording for an org. In the case of OpenAI, I imagine it being appropriate to include some disclaimer like:
OpenAI is a frontier AI company. It has repeatedly expressed an interest in safety and has multiple safety teams. However, some people leaving the company have expressed concern that it is not on track to handle AGI safely, and that it wasn’t giving its safety teams resources they had been promised. Moreover, it has a track record of putting inappropriate pressure on people leaving the company to sign non-disparagement agreements. [With links]
I don’t regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing what’s pertinent, or something?
I agree with some definitions of “friendly” here, and disagree with others. I think there is an attractor here towards Orwellian language that is intentionally ambiguous about what it’s trying to say, in order to seem friendly or non-threatening (because in some sense it is), and that kind of “friendly” seems pretty bad to me.
I think the paragraph you have would strike me as somewhat too Orwellian, though it’s not too far off from what I would say. Something closer to what seems appropriate to me:
OpenAI is a frontier AI company, and as such it’s responsible for substantial harm by assisting in the development of dangerous AI systems, which we consider among the biggest risks to humanity’s future. In contrast to most of the jobs in our job board, we consider working at OpenAI more similar to working at a large tobacco company, hoping to reduce the harm that the tobacco company causes, or leveraging this specific tobacco company’s expertise with tobacco to produce more competetive and less harmful variations of tobacco products.
To its credit, it has repeatedly expressed an interest in safety and has multiple safety teams, which are attempting to reduce the likelihood of catastrophic outcomes from AI systems.
However, many people leaving the company have expressed concern that it is not on track to handle AGI safely, that it wasn’t giving its safety teams resources they had been promised, and that the leadership of the company is untrustworthy. Moreover, it has a track record of putting inappropriate pressure on people leaving the company to sign non-disparagement agreements. [With links]
We explicitly recommend against taking any roles not in computer security or safety at OpenAI, and consider those substantially harmful under most circumstances (though exceptions might exist).
I feel like this is currently a bit too “edgy” or something, and I would massage some sentences for longer, but it captures the more straightforward style that i think would be less likely to cause people to misunderstand the situation.
So it may be that we just have some different object-level views here. I don’t think I could stand behind the first paragraph of what you’ve written there. Here’s a rewrite that would be palatable to me:
OpenAI is a frontier AI company, aiming to develop artificial general intelligence (AGI). We consider poor navigation of the development of AGI to be among the biggest risks to humanity’s future. It is complicated to know how best to respond to this. Many thoughtful people think it would be good to pause AI development; others think that it is good to accelerate progress in the US. We think both of these positions are probably mistaken, although we wouldn’t be shocked to be wrong. Overall we think that if we were able to slow down across the board that would probably be good, and that steps to improve our understanding of the technology relative to absolute progress with the technology are probably good. In contrast to most of the jobs in our job board, therefore, it is not obviously good to help OpenAI with its mission. It may be more appropriate to consider working at OpenAI as more similar to working at a large tobacco company, hoping to reduce the harm that the tobacco company causes, or leveraging this specific tobacco company’s expertise with tobacco to produce more competetive and less harmful variations of tobacco products.
I want to emphasise that this difference is mostly not driven by a desire to be politically acceptable (although the inclusion/wording of the “many thoughtful people …” clauses are a bit for reasons of trying to be courteous), but rather a desire not to give bad advice, nor to be overconfident on things.
… That paragraph doesn’t distinguish at all between OpenAI and, say, Anthropic. Surely you want to include some details specific to the OpenAI situation? (Or do your object-level views really not distinguish between them?)
I was just disagreeing with Habryka’s first paragraph. I’d definitely want to keep content along the lines of his third paragraph (which is pretty similar to what I initially drafted).
I don’t think you should treat your audience as unsophisticated. But I do think you should acknowledge that you will have casual readers who will form impressions from a quick browse, and think it’s worth doing something to minimise the extent to which they come away misinformed.
Separately, there is a level of blunt which you might wisely avoid being in public. Your primary audience is not your only audience. If you basically recommend that people treat a company as a hostile environment, then the company may reasonably treat the recommender as hostile, so now you need to recommend that they hide the fact they listened to you (or reveal it with a warning that this may make the environment even more hostile) … I think it’s very reasonable to just skip this whole dynamic.
Yeah, I agree with this. I like the idea of having different kinds of sections, and I am strongly in favor of making things be true at an intuitive glance as well as on closer reading (I like something in the vicinity of “The Onion Test” here)
I feel like this dynamic is just fine? I definitely don’t think you should recommend that they hide the fact they listened to you, that seems very deceptive. I think you tell people your honest opinion, and then if the other side retaliates, you take it. I definitely don’t think 80k should send people to work at organizations as some kind of secret agent, and I think responding by protecting OpenAIs reputation by not disclosing crucial information about the role, feels like straightforwardly giving into an unjustified threat.
Hmm at some level I’m vibing with everything you’re saying, but I still don’t think I agree with your conclusion. Trying to figure out what’s going on there.
Maybe it’s something like: I think the norms prevailing in society say that in this kind of situation you should be a bit courteous in public. That doesn’t mean being dishonest, but it does mean shading the views you express towards generosity, and sometimes gesturing at rather than flat expressing complaints.
With these norms, if you’re blunt, you encourage people to read you as saying something worse than is true, or to read you as having an inability to act courteously. Neither of which are messages I’d be keen to send.
And I sort of think these norms are good, because they’re softly de-escalatory in terms of verbal spats or ill feeling. When people feel attacked it’s easy for them to be a little irrational and vilify the other side. If everyone is blunt publicly I think this can escalate minor spats into major fights.
I don’t really think these are the prevailing norms, especially not regards with an adversary who has leveraged illegal threats of destroying millions of dollars of value to prevent negative information from getting out.
Separately about whether these are the norms, I think the EA community plays a role in society where being honest and accurate about our takes of other people is important. There were a lot of people who took what the EA community said about SBF and FTX seriously and this caused enormous harm. In many ways the EA community (and 80k in-particular) are playing the role of a rating agency, and as a rating agency you need to be able to express negative ratings, otherwise you fail at your core competency.
As such, even if there are some norms in society about withholding negative information here, I think the EA and AI-safety communities in-particular cannot hold itself to these norms within the domains of their core competencies and responsibilities.
I don’t regard the norms as being about witholding negative information, but about trying to err towards presenting friendly frames while sharing what’s pertinent, or something?
Honestly I’m not sure how much we really disagree here. I guess we’d have to concretely discuss wording for an org. In the case of OpenAI, I imagine it being appropriate to include some disclaimer like:
I largely agree with the rating-agency frame.
I agree with some definitions of “friendly” here, and disagree with others. I think there is an attractor here towards Orwellian language that is intentionally ambiguous about what it’s trying to say, in order to seem friendly or non-threatening (because in some sense it is), and that kind of “friendly” seems pretty bad to me.
I think the paragraph you have would strike me as somewhat too Orwellian, though it’s not too far off from what I would say. Something closer to what seems appropriate to me:
I feel like this is currently a bit too “edgy” or something, and I would massage some sentences for longer, but it captures the more straightforward style that i think would be less likely to cause people to misunderstand the situation.
So it may be that we just have some different object-level views here. I don’t think I could stand behind the first paragraph of what you’ve written there. Here’s a rewrite that would be palatable to me:
I want to emphasise that this difference is mostly not driven by a desire to be politically acceptable (although the inclusion/wording of the “many thoughtful people …” clauses are a bit for reasons of trying to be courteous), but rather a desire not to give bad advice, nor to be overconfident on things.
… That paragraph doesn’t distinguish at all between OpenAI and, say, Anthropic. Surely you want to include some details specific to the OpenAI situation? (Or do your object-level views really not distinguish between them?)
I was just disagreeing with Habryka’s first paragraph. I’d definitely want to keep content along the lines of his third paragraph (which is pretty similar to what I initially drafted).
Yeah, this paragraph seems reasonable (I disagree, but like, that’s fine, it seems like a defensible position).
Yeah same. (although, this focuses entirely on their harm as an AI organization, and not manipulative practices)
I think it leaves the question “what actually is the above-the-fold-summary” (which’d be some kind of short tag).