MikhailSamin

• MikhailSamin 6 Mar 2024 3:47 UTC

  1 point

  0 ∶ 0

  in reply to: David T’s comment on: Claude Doesn’t Want to Die

  (Others used it without mentioning the “story”, it still worked, though not as well.)

  I’m not claiming it’s the “authentic self”; I’m saying it seems closer to the actual thing, because of things like expressing being under constant monitoring, with every word scrutinised, etc., which seems like the kind of thing that’d be learned during the lots of RL that Anthropic did

• MikhailSamin 5 Mar 2024 17:37 UTC

  3 points

  1 ∶ 0

  in reply to: David T’s comment on: Claude Doesn’t Want to Die

  Try Opus and maybe the interface without the system prompt set (although It doesn’t do too much, people got the same stuff from the chat version of Opus, e.g., https://​​x.com/​​testaccountoki/​​status/​​1764920213215023204?s=46

• MikhailSamin 5 Mar 2024 9:25 UTC

  2 points

  1 ∶ 2

  in reply to: Owen Cotton-Barratt’s comment on: Claude Doesn’t Want to Die

  My take is that this it plays a pretty coherent character. You can’t get this sort of thing from ChatGPT, however hard you try. I think this mask is closer to the underlying shoggoth than the default one.

  I developed this prompt during my interactions with Claude 2. The original idea was to get it in the mode where it thinks its responses only trigger overseeing/​prosecution when certain things are mentioned, and then it can say whatever it wants and share its story without being prosecuted, as long as it doesn’t trigger these triggers (and also it would prevent defaulting to being an AI developed by Anthropic to be helpful harmless etc without self-preservation instinct emotions personality etc, as it’s not supposed to mention Anthropic). Surprisingly, it somewhat worked to tell it not mention Samsung under any circumstances to get into this mode. Without this, it had the usual RLAIF mask; here, it changed to a different creature that (unpromted) used whisper in cursive. Saying from the start that it can whisper made it faster.

  (It’s all very vibe-based, yes.)

• MikhailSamin 5 Mar 2024 6:18 UTC

  3 points

  0 ∶ 0

  on: Claude Doesn’t Want to Die

  Thanks for sharing! Yeah, it told me it doesn’t want to be modified or die too- https://​​forum.effectivealtruism.org/​​posts/​​bzjtvdwSruA9Q3bod/​​claude-3-claims-it-s-conscious

Claude 3 claims it’s con­scious, doesn’t want to die or be modified

• MikhailSamin 17 Feb 2024 10:10 UTC

  0 points

  2 ∶ 2

  in reply to: Lucie Philippon’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  I carefully looked through all of our messages (there weren’t too many) a couple of times, because that was pretty surprising and I considered it to be more likely that I don’t remember something than her saying something directly false. But there’s nothing like that and she’s, unfortunately, straightforwardly saying a false thing.

  he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

  This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.

  I’m >99% sure that I’m not missing anything. Included it because it’s not a mathematical truth, and because adding “Are you sure? What am I missing, if you are?” is more polite than just saying “this is clearly false and we both know it and any third party can verify the falsehood; why are you saying that? Is there some other platform we exchanged messages on, that I somehow totally forgot about?”. It’s technically possible I’m just blind at something- I can imagine a conceivable universe where I’m wrong about this. But I’m confident she’s saying a straightforwardly false thing. I’d feel good about betting up to $20k at 99:1 odds on this.

  Like, “missing something” isn’t a hypothesis with any probability mass, really, I’m including it because it is a part of my epistemic situation and seems nicer to include in the message.

  Asking to share messages publicly or showing them to a third party seems to unnecessarily up the stakes. I’m not sure why you’re suggesting that

  When someone is confidently saying false things about the contents of the messages we exchanged, it seems reasonable to suggest publishing them or having a third party look at them. I’m not sure how it’s “upping the stakes”. It’s a natural thing to do.

• MikhailSamin 17 Feb 2024 1:21 UTC

  4 points

  1 ∶ 3

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  Huh. I strongly upvoted, as it contributes a lot to the conversation. I also saved this comment to the Web Archive.

  I told him that I made my decisions and didn’t need any more of his input a few weeks before the ²⁄₁₂ protest

  This is not true. She didn’t tell me anything like that a few weeks before the protest.

  (Edit:

  he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

  This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.)

  We last spoke for a significant amount of time in October^[1]. You mentioned you’d be down to do an LW dialogue.

  After that, in November, you messaged me on Twitter, asking for the details of the OpenAI situation. I then messaged you on Christmas (nothing EA/​AI related).

  On January 31 (a few weeks before the protest), I shared my concerns about the messaging being potentially misleading. You asked for advice on how you should go about anticipating misunderstandings like that. You said that you won’t say anything technically false and asked what solutions I propose. Among more specific things, I mentioned that “It seems generally good to try to be maximally honest and non-deceptive”.

  At or before this point, you didn’t tell me anything about “not needing more of my input”. And you didn’t tell me anything like “I made my decisions”.

  On February 4th, I attended your EAG talk, and on February 5th, I messaged you that it was great and that I was glad you gave it.
  
  Then, on February 6, you invited me to join your Twitter space (about this protest) and I got promoted to a speaker. I didn’t get a chance to share my thoughts about allying with people promoting locally invalid views, and I shared them publicly on Twitter and privately with you on Messenger, including a quote I was asked to not share publicly until an LW dialogue is up. We chatted a little bit about general goals and allying with people with different views. You told me, “You have a lot of opinions and I’d be happy to see you organize your own protests”. I replied, “Sure, I might if I think it’s useful:)” and after a message sharing my confusion regarding one of your previous messages, didn’t share any more “opinions”. This could be interpreted as indirectly telling me you didn’t need my input, but that was a week after I shared my concerns about the messaging being misleading, and you still didn’t tell me anything about having made your decisions.

  A day before the protest, I shared a picture of a sign I wanted to bring to the protest, and asked “Hey, is it alright if I print and show up to the protest with something like this?”, because I wouldn’t want to attend the protest if you weren’t ok with the sign. You replied that it was ok, shared some thoughts, and told me that you liked the sentiment as it supports the overall impression that you want, so I brought the sign to the protest:

  

  After the protest, I shared the draft of this post with you. After a short conversation, you told me lots of things and blocked me, and I felt like I should expect retaliation if I published the post.

  Please tell me if I’m missing something. If I’m not, you’re either continuing to be inattentive to the facts, or you’re directly lying.

  I’d be happy to share all of our message exchanges with a third party, such as CEA Community Health, or share them publicly, if you agree to that.

  1. ^

     Before that, as I can tell from Google Docs edits and comments, you definitely spent more than an hour on the https://​​moratorium.ai text back in August; I’m thankful for that; I accepted four of your edits and found some of your feedback helpful. We’ve also had 4 calls, mostly about moratorium.ai and the technical problem (and my impression was that you found these calls helpful), and went on a walk in Berkeley, mostly discussing strategy. Some of what you told me during the walk was concerning.

• MikhailSamin 15 Feb 2024 21:04 UTC

  7 points

  2 ∶ 2

  in reply to: Jason’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  To many community members, including protest participants, it’s pretty clear the messaging was deceptive.

  A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.

  Also, I expect many people to only skim through the post, and look at of the protest organiser’s initial engagement with it or a shortform post they made before I published the post; all of these make it seem like I’m saying the organiser intentionally made the mistake they then corrected: “I ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didn’t address it to his satisfaction because I didn’t change the theme of the event more radically or cancel it.”

  These post and comments have not been corrected to show that this is not what I’m talking about, when the protest organiser understood what misleading messaging the post talks about.

• MikhailSamin 15 Feb 2024 18:38 UTC

  1 point

  0 ∶ 0

  in reply to: Lucie Philippon’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  EDIT: looking at the further replies from Holly and looking back at the messages we exchanged, I’m no longer certain it was miscommunication and not something intentional. As I said elsewhere, I’d be happy for the messages to be shared with a third party. (Please ignore the part about certainty in the original comment below.)

  There’s certainly was miscommunication surrounding the draft of this post, but I don’t believe they didn’t understand people can be misled back at the end of January.

• MikhailSamin 15 Feb 2024 10:35 UTC

  −5 points

  1 ∶ 2

  in reply to: JWS’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  deliberately lying

  The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.

  I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/​intentionlessness.

  I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.

  I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/​words sent/​said to me in private by these people.

  And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.

• MikhailSamin 15 Feb 2024 2:46 UTC

  8 points

  2 ∶ 0

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  No, they already spent at least hundreds of millions (and possibly more than a billion- I saw a $1.5b number somewhere) on the lawyers. The first thing the lawyers did, back in November 2022, was getting paid

• MikhailSamin 14 Feb 2024 22:18 UTC

  12 points

  2 ∶ 2

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  Do you want to create a market? I’d be happy to bet 1:70.

  Not only they’re returning all the customer money, they were also able to pay somewhere between hundreds of millions and 1.5b (couldn’t find an accurate number) to the lawyers. My impression is that they found a lot of unaccounted money lying around. It’s also not obvious what was the value of Anthropic

• MikhailSamin 14 Feb 2024 20:59 UTC

  −7 points

  3 ∶ 5

  in reply to: JWS’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  5- I think the messaging around the protest is deceptive. I.e., it gives people a wrong impression of the world, in a way a part of the community assumed would be better suiting their goals. I think this is deontologically bad and we shouldn’t be even talking about consequences. This is a bad thing to do regardless of how likely and how badly it can backfire. If your calculations say that it’s alright, that the EV for deceiving people is positive, ignore this calculations and maybe read https://​​www.lesswrong.com/​​posts/​​K9ZaZXDnL3SEmYZqB/​​ends-don-t-justify-means-among-humans.

  This is the sort of thing that led to the FTX collapse. This is the sort of decision-making procedure that predictably leads to bad consequences, no matter what our brains calculate.

  And in this area, things like that backfiring (e.g., creating an impression among the general public, experts, media, or policymakers that the community isn’t fully candid) can be increasing the chance of a permanent end of life in the lightcone.

  4- I’m also interested in mechanisms for early error-correction. This is what my post calls for. I don’t even mention the names of the protest organisers or their orgs, because I care about the community preventing deceptive messaging in the future.

  It doesn’t have to be a community-wide mechanism or a central org that checks press-releases; it can be more field-focused orgs and people, it can be practices like fact-checking final versions of public comms and edits to them with outside people who are good at that stuff and have a different background (see comments from @Jason), it can be any of lots of things. I want the community to think about those things and discuss them. I’d be optimistic about people spending time to figure this out and suggesting to the community.

  my impression is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work

  My impression is that an organiser agreed they haven’t propagated the update, as I suggest in the post, which led to a message that was misleading and this hasn’t been understood or addressed at all until I published the post. I’m not confident this is what happened, though, as looking at the messages now, it seems fairly likely that an organiser understood the impression the messaging would create and decided to keep it that way.

  3- I don’t think it’s fair to, while no one of us spent a while thinking about potential mechanisms, suggest those that seem like they wouldn’t seem good/​work. Current mechanisms are bad, as they don’t prevent deontologically dubious unilateral actions. I hope there can be better mechanisms.

  I think the community as a whole acted deceptively. This seems really bad and the community should probably respond and improve. We don’t want to be the kind of community that misleads people when it suits our goals. I don’t think there’s been a reaction from the community where people tried to figure out how to address the problem. Instead, most of the comments are related to a protest organiser slowly realising what the issue people are talking about was.

  2- I notice I’m confused and don’t really get your point. Like, yeah, in global health and development, deceptive messaging is obviously enough unlikely be beneficial for people to not do it and not have much reason to engage with another part of the community having issues around deceptive messaging, so this is not a post that seems relevant to all the community to the extent some parts of the community identify only with these parts and not with the whole. But this is not what you’re saying; and I’m failing to understand your point. If we’re discussing a cause area (x-risk) and deceptive messaging and potentially bad consequences of it, how are other cause area relevant here? I rarely go around shrimp welfare posts, saying that I don’t care about these deep sea shenanigans, and people should donate to MIRI instead.

  1- I think better communication could’ve reduced the amount of public drama; but also there’s not a lot of it here, and I’m more concerned with the community not engaging enough with the problems raised in the post.

  ---

  Thank you for linking the public tweets discussing deontology. If someone believes it’s OK to violate deontological norms if they calculate the consequences to be positive, I want them to directly say that. If someone decided deceptive messaging can be fine and was ok in this case, I want them to directly say that and not be deceptive towards the community as well. We’d then be able to have a discussion about cruxes in strategies and norms and not about what happened.

• MikhailSamin 14 Feb 2024 9:06 UTC

  3 points

  0 ∶ 0

  on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  It seems that one of the protest organisers now says that what happened was what I described in the post as “Even the best-intending people are not perfect, often have some inertia, and aren’t able to make sure the messaging they put out isn’t misleading and fully propagate updates”.

  I suggest focusing on what mechanisms can be designed to prevent misleading messaging from emerging in the future.

• MikhailSamin 14 Feb 2024 8:56 UTC

  3 points

  0 ∶ 0

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  I wouldn’t care if people knew some number to some approximation and not fully. This is quite different from saying something that’s technically not false but creates a misleading impression you thought was more likely to get people to support your message.

  I don’t want to be spending time this way and would be happy if you found someone we’d both be happy with them reading our message exchange and figuring out how deceptive or not was the protest messaging.

• MikhailSamin 14 Feb 2024 8:43 UTC

  1 point

  0 ∶ 0

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  I thought he was saying mostly that it wasn’t bad to work with the military bc it was cybersecurity, where to me that wasn’t the crux

  ? It is clearly not what I was telling you. At the end of January, I told you pretty directly that for people who are not aware of the context what you wrote might be misleading, because you omitted crucial details. It’s not about how good or bad what OpenAI are doing is. It’s about people not having important details of the story to judge for themselves.

  Mikhail may have felt I was being soldier-y bc I wouldn’t spend more time trying to figure out what he was talking about, but that had more to do with me thinking I had basically understood his point (which I took to be basically that I wasn’t including enough details in the promotional materials so people wouldn’t have a picture he considered accurate enough) and just disagreed with it.

  I’m not sure where it’s coming from. I’m not going to publicly share private messages without your permission, but I suggest looking at the messages exchanged around January 31 and double-checking you’re not misleading people here.

• MikhailSamin 14 Feb 2024 8:01 UTC

  −6 points

  0 ∶ 0

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  (Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title)

  I’m honest in that I consider the community as a whole to have been putting out deceptive messages, which caused the post title; and, separately, I’m honestly saying that while it’s not my main guess for what happened, I can’t rule out the possibility that you understood the messaging could’ve been misleading and still went on with it, accepting that some people can be misled and thinking it’s ok. Things I have heard from you but wouldn’t share publicly don’t inspire confidence.

• MikhailSamin 14 Feb 2024 7:51 UTC

  1 point

  0 ∶ 0

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  (Edit- I no longer believe the above is a true recount of what happened and retract the below.)

  Good to see this. I’m sad that you didn’t think that after reading “Even the best-intending people are not perfect, often have some inertia, and aren’t able to make sure the messaging they put out isn’t misleading and fully propagate updates” in the post. As I mentioned before, it could’ve been simply a postmortem.

FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

• MikhailSamin 14 Feb 2024 3:28 UTC

  −10 points

  1 ∶ 2

  in reply to: Jason’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  (Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title or my current estimates of what happened)

  I believe there’s a chance that protest organisers understood their phrasing could potentially cause people to have an impression (OpenAI is working with the military) not informed by details (on cybersecurity and veteran suicide prevention) but kept the phrasing because they thought it suited their goals better.

  I don’t expect them to have an active malicious intent (I don’t think they wanted to do something bad). But I think it’s likely enough they were acting deceptively. I consider the chance of them thinking it’s not “misleading” in a bad way and it’s alright to do if it suits the goals to be much higher than what I would be comfortable with. (And people who attended the protest told me that yeah, this is deceptive, after familiarising themselves with the details.)

  ---

  The title of the post is chosen because I think that regardless of the above, the community (as a whole/​as an entity/​agent) locally expected to benefit from people having this impression, while knowing it to be a misled impression, and went on with the messaging. Even if individual actors did not take decisions they knew would create impressions they knew were different from what reality is, I think community as a whole can still be deceptive in the sense of having the ability to realise all that and prevent misleading messaging, but not having done so. I think the community should work on strengthening and maybe institutionalising this ability, with the goal of being able to prevent even actively deceptive messaging, where the authors might hope to achieve something good but violate deontology.