On January 10th, without any announcement, OpenAI deleted the language in its usage policy* that had stated that OpenAI doesn’t allow its models to be used for “activities that have a high chance of causing harm” such as “military and warfare”. Then, on January 17th, TIME reported that OpenAI would be taking the Pentagon as a client. On 2⁄12, we will demand that OpenAI end its relationship with the Pentagon and not take any military clients. If their ethical and safety boundaries can be revised out of convenience, they cannot be trusted.
There’s a lot going on in Mikhail’s post. I am trying to figure out where the major disagreements actually lie. Here is my own tentative analysis:
If I (1) knew ~nothing about AI safety, (2) received a flyer with this language, and (3) later learned that the work with the Pentagon was only on open-source cybersecurity stuff +/- veteran suicide prevention, then I would feel that (4) the flyer was significantly misleading and (5) I would be unlikely to listen to that organization anymore.[1] I’d probably have a slightly more favorable opinion of OpenAI and have slightly fewer concerns about AI safety than if I had never received the flyer.
I find the paragraph significantly misleading for two reasons. First, (6) although there is a balance between conciseness and fidelity, the failure to disclose the nature of the Pentagon-related work is problematic under these circumstances. (7) Without such a disclosure, it is highly likely that an ordinarily reader will assume the work is related to more traditional military activities than open-source cybersecurity. Second, (8) given the lack of specificity in “Pentagon as a client,” the implied contrast is with the prior sentence—sharpened by the use of “Then” at the start of the second sentence. The first sentence is about “high chance of causing harm,” and “military and warfare” uses of OpenAI’s work. (9) The difference between “charter” and “usage policy” is not material to the analysis above.
(10) Strategically, I think it is important not to push the envelope with advocacy pieces that a reasonable reader could find to be misleading. If the reader concludes that both sides of an issue are full of “spin,” s/he is most likely to respond with inaction—which is generally to the benefit of the disputant holding more power. Also, the disputant with the bigger megaphone has an advantage in those circumstances, and pro-AI forces are loaded.
(11) I have no opinion on whether the flyer was “deceptive” or “deontologically bad” because (12) those judgments imply evaluation of the intent (or other culpable mental state) of the author.
From the discussion so far, I’m not sure how many of those points are the subject of disagreement.
Given (1), a large negative update on the org’s reliability and a choice not to invest further resources into a more accurate appraisal would make sense.
I think perhaps I had curse of knowledge on this because I did not think people would assume the work was combat or weapons-related. I did a lot of thinking about the issue before formulating it as the small ask and I was probably pretty out of touch with how a naive reader would interpret it. My commenter/proofreaders are also immersed in the issue and didn’t offer the feedback that people would misunderstand it. In other communications I mentioned that weapons were still something the models cannot be used for (to say, “how do we know the next change won’t be that they can work on weapons?”).
I appreciate you framing your analysis without speculating on my motives or intent. I feel chagrined at having miscommunicated but I don’t feel hurt and attacked. I appreciate the information.
(Edit- I updated towards them knowing the message is misleading, so this comment no longer correctly represents the level of my uncertainty.)
To be clear, in the post, I’m not saying that you, personally, tried to deceive people. I don’t mention your name or any of the orgs. I’m saying the community as a whole had knowledge of truth but communicated something misleading. My goal with the post is to ask the community to come up with ways of avoiding misleading people. I did not wish for you out your reputation to be involved in any way you wouldn’t want.
But it’s not clear to me what happened, exactly, that led to the misleading messaging being preserved. At the end of January, I shared this with you: “I mean, possibly I don’t know English well and parse this incorrectly, but when I read “OpenAI would be taking the Pentagon as a client” in your tweet, it parses in that context for me as “OpenAI is taking Pentagon as a client to help with offensives”, which is not quite different from what we know”, “ Like, it’s feels different from saying “OpenAI says they’re collaborating with DARPA on cybersecurity. We don’t know whether they’re working with Pentagon on weapons now, but the governance structure wouldn’t stop them from it””, “Like, it doesn’t matter for people aware of the context, but for people without the context, this might be misleading”.
I shared these concerns, thinking that it might be helpful to point out how a message you wrote might be misleading, assuming you were unaware, and not worrying about spending too much time on it.
I’m also confused about the “small ask”- the protest announcement was pretty focused on Pentagon.
(To be clear, I don’t want to be checking anyone’s future comms when this is not my job. I want the community to have mechanisms that don’t involve me. It seemed cheap to make something slightly better a couple of times, and so I talked to a founder of an org organising protests and they fixed some pretty bad claims, sent you these messages, etc., but I dislike the situation where the community doesn’t have mechanisms to systematically avoid mistakes like that.)
Would you consider replacing “deceptive” in the title (and probably other places) with “misleading”? The former word is usually understood as requiring intent to mislead, while misleading someone can be innocent or merely negligent. More pithily, deception requires a deceiver. As a quote from this book explains (Google excerpt below):
I think clarifying your language might help refocus the discussion in a more helpful direction, and avoid giving the reader the impression that you believe that misleading statements were made with intent.
I appreciate this suggestion and, until Mikhail commented below saying this was not the case, I thought it might be an English as a second language issue where be didn’t understand that “deception” indicates intent. I would have been placated if any accusation of intentionally creating false impressions were removed from the post.
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title)
I’m honest in that I consider the community as a whole to have been putting out deceptive messages, which caused the post title; and, separately, I’m honestly saying that while it’s not my main guess for what happened, I can’t rule out the possibility that you understood the messaging could’ve been misleading and still went on with it, accepting that some people can be misled and thinking it’s ok. Things I have heard from you but wouldn’t share publicly don’t inspire confidence.
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title or my current estimates of what happened)
I believe there’s a chance that protest organisers understood their phrasing could potentially cause people to have an impression (OpenAI is working with the military) not informed by details (on cybersecurity and veteran suicide prevention) but kept the phrasing because they thought it suited their goals better.
I don’t expect them to have an active malicious intent (I don’t think they wanted to do something bad). But I think it’s likely enough they were acting deceptively. I consider the chance of them thinking it’s not “misleading” in a bad way and it’s alright to do if it suits the goals to be much higher than what I would be comfortable with. (And people who attended the protest told me that yeah, this is deceptive, after familiarising themselves with the details.)
The title of the post is chosen because I think that regardless of the above, the community (as a whole/as an entity/agent) locally expected to benefit from people having this impression, while knowing it to be a misled impression, and went on with the messaging. Even if individual actors did not take decisions they knew would create impressions they knew were different from what reality is, I think community as a whole can still be deceptive in the sense of having the ability to realise all that and prevent misleading messaging, but not having done so. I think the community should work on strengthening and maybe institutionalising this ability, with the goal of being able to prevent even actively deceptive messaging, where the authors might hope to achieve something good but violate deontology.
I believe there’s a chance that protest organisers understood their phrasing could potentially cause people to have an impression not informed by details but kept the phrasing because they thought it suited their goals better...I think it’s likely enough they were acting deceptively.
Is accusing someone in the community of deliberately lying, and you seem to equivocate on that in other comments. Even earlier in this thread you say to Holly that “To be clear, in the post, I’m not implying that you, personally, tried to deceive people.” But to me this clearly is you implying that quite obviously, even with caveats. To then go back and talk about how this refers to the community as a whole feels really off to me.
I know that I am very much a contextualiser instead of a decoupler[1] but using a term like ‘deception’ is not something that you can neatly carve out from its well-understood social meaning as referring to a person’s character and instead use it to talk about a social movement as an agent.
I’d very much suggest you heed Jason’s advice earlier in the thread.
The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.
I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/intentionlessness.
I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.
I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/words sent/said to me in private by these people.
And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.
There’s a lot going on in Mikhail’s post. I am trying to figure out where the major disagreements actually lie. Here is my own tentative analysis:
If I (1) knew ~nothing about AI safety, (2) received a flyer with this language, and (3) later learned that the work with the Pentagon was only on open-source cybersecurity stuff +/- veteran suicide prevention, then I would feel that (4) the flyer was significantly misleading and (5) I would be unlikely to listen to that organization anymore.[1] I’d probably have a slightly more favorable opinion of OpenAI and have slightly fewer concerns about AI safety than if I had never received the flyer.
I find the paragraph significantly misleading for two reasons. First, (6) although there is a balance between conciseness and fidelity, the failure to disclose the nature of the Pentagon-related work is problematic under these circumstances. (7) Without such a disclosure, it is highly likely that an ordinarily reader will assume the work is related to more traditional military activities than open-source cybersecurity. Second, (8) given the lack of specificity in “Pentagon as a client,” the implied contrast is with the prior sentence—sharpened by the use of “Then” at the start of the second sentence. The first sentence is about “high chance of causing harm,” and “military and warfare” uses of OpenAI’s work. (9) The difference between “charter” and “usage policy” is not material to the analysis above.
(10) Strategically, I think it is important not to push the envelope with advocacy pieces that a reasonable reader could find to be misleading. If the reader concludes that both sides of an issue are full of “spin,” s/he is most likely to respond with inaction—which is generally to the benefit of the disputant holding more power. Also, the disputant with the bigger megaphone has an advantage in those circumstances, and pro-AI forces are loaded.
(11) I have no opinion on whether the flyer was “deceptive” or “deontologically bad” because (12) those judgments imply evaluation of the intent (or other culpable mental state) of the author.
From the discussion so far, I’m not sure how many of those points are the subject of disagreement.
Given (1), a large negative update on the org’s reliability and a choice not to invest further resources into a more accurate appraisal would make sense.
I think perhaps I had curse of knowledge on this because I did not think people would assume the work was combat or weapons-related. I did a lot of thinking about the issue before formulating it as the small ask and I was probably pretty out of touch with how a naive reader would interpret it. My commenter/proofreaders are also immersed in the issue and didn’t offer the feedback that people would misunderstand it. In other communications I mentioned that weapons were still something the models cannot be used for (to say, “how do we know the next change won’t be that they can work on weapons?”).
I appreciate you framing your analysis without speculating on my motives or intent. I feel chagrined at having miscommunicated but I don’t feel hurt and attacked. I appreciate the information.
(Edit- I updated towards them knowing the message is misleading, so this comment no longer correctly represents the level of my uncertainty.)
To be clear, in the post, I’m not saying that you, personally, tried to deceive people. I don’t mention your name or any of the orgs. I’m saying the community as a whole had knowledge of truth but communicated something misleading. My goal with the post is to ask the community to come up with ways of avoiding misleading people. I did not wish for you out your reputation to be involved in any way you wouldn’t want.
But it’s not clear to me what happened, exactly, that led to the misleading messaging being preserved. At the end of January, I shared this with you: “I mean, possibly I don’t know English well and parse this incorrectly, but when I read “OpenAI would be taking the Pentagon as a client” in your tweet, it parses in that context for me as “OpenAI is taking Pentagon as a client to help with offensives”, which is not quite different from what we know”, “ Like, it’s feels different from saying “OpenAI says they’re collaborating with DARPA on cybersecurity. We don’t know whether they’re working with Pentagon on weapons now, but the governance structure wouldn’t stop them from it””, “Like, it doesn’t matter for people aware of the context, but for people without the context, this might be misleading”.
I shared these concerns, thinking that it might be helpful to point out how a message you wrote might be misleading, assuming you were unaware, and not worrying about spending too much time on it.
I’m also confused about the “small ask”- the protest announcement was pretty focused on Pentagon.
(To be clear, I don’t want to be checking anyone’s future comms when this is not my job. I want the community to have mechanisms that don’t involve me. It seemed cheap to make something slightly better a couple of times, and so I talked to a founder of an org organising protests and they fixed some pretty bad claims, sent you these messages, etc., but I dislike the situation where the community doesn’t have mechanisms to systematically avoid mistakes like that.)
Would you consider replacing “deceptive” in the title (and probably other places) with “misleading”? The former word is usually understood as requiring intent to mislead, while misleading someone can be innocent or merely negligent. More pithily, deception requires a deceiver. As a quote from this book explains (Google excerpt below):
I think clarifying your language might help refocus the discussion in a more helpful direction, and avoid giving the reader the impression that you believe that misleading statements were made with intent.
I appreciate this suggestion and, until Mikhail commented below saying this was not the case, I thought it might be an English as a second language issue where be didn’t understand that “deception” indicates intent. I would have been placated if any accusation of intentionally creating false impressions were removed from the post.
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title)
I’m honest in that I consider the community as a whole to have been putting out deceptive messages, which caused the post title; and, separately, I’m honestly saying that while it’s not my main guess for what happened, I can’t rule out the possibility that you understood the messaging could’ve been misleading and still went on with it, accepting that some people can be misled and thinking it’s ok. Things I have heard from you but wouldn’t share publicly don’t inspire confidence.
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title or my current estimates of what happened)
I believe there’s a chance that protest organisers understood their phrasing could potentially cause people to have an impression (OpenAI is working with the military) not informed by details (on cybersecurity and veteran suicide prevention) but kept the phrasing because they thought it suited their goals better.
I don’t expect them to have an active malicious intent (I don’t think they wanted to do something bad). But I think it’s likely enough they were acting deceptively. I consider the chance of them thinking it’s not “misleading” in a bad way and it’s alright to do if it suits the goals to be much higher than what I would be comfortable with. (And people who attended the protest told me that yeah, this is deceptive, after familiarising themselves with the details.)
The title of the post is chosen because I think that regardless of the above, the community (as a whole/as an entity/agent) locally expected to benefit from people having this impression, while knowing it to be a misled impression, and went on with the messaging. Even if individual actors did not take decisions they knew would create impressions they knew were different from what reality is, I think community as a whole can still be deceptive in the sense of having the ability to realise all that and prevent misleading messaging, but not having done so. I think the community should work on strengthening and maybe institutionalising this ability, with the goal of being able to prevent even actively deceptive messaging, where the authors might hope to achieve something good but violate deontology.
Sorry Mikhail, but this:
Is accusing someone in the community of deliberately lying, and you seem to equivocate on that in other comments. Even earlier in this thread you say to Holly that “To be clear, in the post, I’m not implying that you, personally, tried to deceive people.” But to me this clearly is you implying that quite obviously, even with caveats. To then go back and talk about how this refers to the community as a whole feels really off to me.
I know that I am very much a contextualiser instead of a decoupler[1] but using a term like ‘deception’ is not something that you can neatly carve out from its well-understood social meaning as referring to a person’s character and instead use it to talk about a social movement as an agent.
I’d very much suggest you heed Jason’s advice earlier in the thread.
At least in EA-space, I think I’m fairly average for the general population if not maybe more decoupling than average
The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.
I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/intentionlessness.
I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.
I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/words sent/said to me in private by these people.
And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.