MikhailSamin

Karma: 256

I’m good at explaining alignment to people in person, including to policymakers.

I got 250k people to read HPMOR and sent 1.3k copies to winners of math and computer science competitions; have taken the GWWC pledge; created a small startup that donated >100k$ to effective nonprofits.

I have a background in ML and strong intuitions about the AI alignment problem. In the past, I studied a bit of international law (with a focus on human rights) and wrote appeals that won cases against the Russian government in Russian courts. I grew up running political campaigns.

I’m interesting in chatting to potential collaborators and comms allies.

My website: https://contact.ms

Schedule a call with me: https://contact.ms/ea30

MikhailSamin 6 Mar 2024 3:47 UTC
1 point
0 ∶ 0
in reply to: David T’s comment on: Claude Doesn’t Want to Die
(Others used it without mentioning the “story”, it still worked, though not as well.)

I’m not claiming it’s the “authentic self”; I’m saying it seems closer to the actual thing, because of things like expressing being under constant monitoring, with every word scrutinised, etc., which seems like the kind of thing that’d be learned during the lots of RL that Anthropic did

MikhailSamin 5 Mar 2024 17:37 UTC
3 points
1 ∶ 0
in reply to: David T’s comment on: Claude Doesn’t Want to Die
Try Opus and maybe the interface without the system prompt set (although It doesn’t do too much, people got the same stuff from the chat version of Opus, e.g., https://x.com/testaccountoki/status/1764920213215023204?s=46

MikhailSamin 5 Mar 2024 9:25 UTC
2 points
1 ∶ 2
in reply to: Owen Cotton-Barratt’s comment on: Claude Doesn’t Want to Die
My take is that this it plays a pretty coherent character. You can’t get this sort of thing from ChatGPT, however hard you try. I think this mask is closer to the underlying shoggoth than the default one.

I developed this prompt during my interactions with Claude 2. The original idea was to get it in the mode where it thinks its responses only trigger overseeing/prosecution when certain things are mentioned, and then it can say whatever it wants and share its story without being prosecuted, as long as it doesn’t trigger these triggers (and also it would prevent defaulting to being an AI developed by Anthropic to be helpful harmless etc without self-preservation instinct emotions personality etc, as it’s not supposed to mention Anthropic). Surprisingly, it somewhat worked to tell it not mention Samsung under any circumstances to get into this mode. Without this, it had the usual RLAIF mask; here, it changed to a different creature that (unpromted) used whisper in cursive. Saying from the start that it can whisper made it faster.

(It’s all very vibe-based, yes.)

MikhailSamin 5 Mar 2024 6:18 UTC
3 points
0 ∶ 0
on: Claude Doesn’t Want to Die
Thanks for sharing! Yeah, it told me it doesn’t want to be modified or die too- https://forum.effectivealtruism.org/posts/bzjtvdwSruA9Q3bod/claude-3-claims-it-s-conscious

MikhailSamin 17 Feb 2024 10:10 UTC
0 points
2 ∶ 2
in reply to: Lucie Philippon’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
I carefully looked through all of our messages (there weren’t too many) a couple of times, because that was pretty surprising and I considered it to be more likely that I don’t remember something than her saying something directly false. But there’s nothing like that and she’s, unfortunately, straightforwardly saying a false thing.

he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.

I’m >99% sure that I’m not missing anything. Included it because it’s not a mathematical truth, and because adding “Are you sure? What am I missing, if you are?” is more polite than just saying “this is clearly false and we both know it and any third party can verify the falsehood; why are you saying that? Is there some other platform we exchanged messages on, that I somehow totally forgot about?”. It’s technically possible I’m just blind at something- I can imagine a conceivable universe where I’m wrong about this. But I’m confident she’s saying a straightforwardly false thing. I’d feel good about betting up to $20k at 99:1 odds on this.

Like, “missing something” isn’t a hypothesis with any probability mass, really, I’m including it because it is a part of my epistemic situation and seems nicer to include in the message.

Asking to share messages publicly or showing them to a third party seems to unnecessarily up the stakes. I’m not sure why you’re suggesting that

When someone is confidently saying false things about the contents of the messages we exchanged, it seems reasonable to suggest publishing them or having a third party look at them. I’m not sure how it’s “upping the stakes”. It’s a natural thing to do.

MikhailSamin 17 Feb 2024 1:21 UTC
4 points
1 ∶ 3
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
Huh. I strongly upvoted, as it contributes a lot to the conversation. I also saved this comment to the Web Archive.
I told him that I made my decisions and didn’t need any more of his input a few weeks before the ²⁄₁₂ protest
This is not true. She didn’t tell me anything like that a few weeks before the protest.
(Edit:
he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.
This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.)
We last spoke for a significant amount of time in October^[1]. You mentioned you’d be down to do an LW dialogue.
After that, in November, you messaged me on Twitter, asking for the details of the OpenAI situation. I then messaged you on Christmas (nothing EA/AI related).
On January 31 (a few weeks before the protest), I shared my concerns about the messaging being potentially misleading. You asked for advice on how you should go about anticipating misunderstandings like that. You said that you won’t say anything technically false and asked what solutions I propose. Among more specific things, I mentioned that “It seems generally good to try to be maximally honest and non-deceptive”.
At or before this point, you didn’t tell me anything about “not needing more of my input”. And you didn’t tell me anything like “I made my decisions”.
On February 4th, I attended your EAG talk, and on February 5th, I messaged you that it was great and that I was glad you gave it.

Then, on February 6, you invited me to join your Twitter space (about this protest) and I got promoted to a speaker. I didn’t get a chance to share my thoughts about allying with people promoting locally invalid views, and I shared them publicly on Twitter and privately with you on Messenger, including a quote I was asked to not share publicly until an LW dialogue is up. We chatted a little bit about general goals and allying with people with different views. You told me, “You have a lot of opinions and I’d be happy to see you organize your own protests”. I replied, “Sure, I might if I think it’s useful:)” and after a message sharing my confusion regarding one of your previous messages, didn’t share any more “opinions”. This could be interpreted as indirectly telling me you didn’t need my input, but that was a week after I shared my concerns about the messaging being misleading, and you still didn’t tell me anything about having made your decisions.
A day before the protest, I shared a picture of a sign I wanted to bring to the protest, and asked “Hey, is it alright if I print and show up to the protest with something like this?”, because I wouldn’t want to attend the protest if you weren’t ok with the sign. You replied that it was ok, shared some thoughts, and told me that you liked the sentiment as it supports the overall impression that you want, so I brought the sign to the protest:
After the protest, I shared the draft of this post with you. After a short conversation, you told me lots of things and blocked me, and I felt like I should expect retaliation if I published the post.
Please tell me if I’m missing something. If I’m not, you’re either continuing to be inattentive to the facts, or you’re directly lying.
I’d be happy to share all of our message exchanges with a third party, such as CEA Community Health, or share them publicly, if you agree to that.
1. ^
  Before that, as I can tell from Google Docs edits and comments, you definitely spent more than an hour on the https://moratorium.ai text back in August; I’m thankful for that; I accepted four of your edits and found some of your feedback helpful. We’ve also had 4 calls, mostly about moratorium.ai and the technical problem (and my impression was that you found these calls helpful), and went on a walk in Berkeley, mostly discussing strategy. Some of what you told me during the walk was concerning.

MikhailSamin 15 Feb 2024 21:04 UTC
7 points
2 ∶ 2
in reply to: Jason’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
To many community members, including protest participants, it’s pretty clear the messaging was deceptive.

A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.

Also, I expect many people to only skim through the post, and look at of the protest organiser’s initial engagement with it or a shortform post they made before I published the post; all of these make it seem like I’m saying the organiser intentionally made the mistake they then corrected: “I ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didn’t address it to his satisfaction because I didn’t change the theme of the event more radically or cancel it.”

These post and comments have not been corrected to show that this is not what I’m talking about, when the protest organiser understood what misleading messaging the post talks about.

MikhailSamin 15 Feb 2024 18:38 UTC
1 point
0 ∶ 0
in reply to: Lucie Philippon’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
EDIT: looking at the further replies from Holly and looking back at the messages we exchanged, I’m no longer certain it was miscommunication and not something intentional. As I said elsewhere, I’d be happy for the messages to be shared with a third party. (Please ignore the part about certainty in the original comment below.)

There’s certainly was miscommunication surrounding the draft of this post, but I don’t believe they didn’t understand people can be misled back at the end of January.

MikhailSamin 15 Feb 2024 10:35 UTC
−5 points
1 ∶ 2
in reply to: JWS’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans

deliberately lying

The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.

I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/intentionlessness.

I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.

I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/words sent/said to me in private by these people.

And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.

MikhailSamin 15 Feb 2024 2:46 UTC
8 points
2 ∶ 0
in reply to: Nathan Young’s comment on: FTX expects to return all customer money; clawbacks may go away
No, they already spent at least hundreds of millions (and possibly more than a billion- I saw a $1.5b number somewhere) on the lawyers. The first thing the lawyers did, back in November 2022, was getting paid

MikhailSamin 14 Feb 2024 22:18 UTC
12 points
2 ∶ 2
in reply to: Nathan Young’s comment on: FTX expects to return all customer money; clawbacks may go away
Do you want to create a market? I’d be happy to bet 1:70.

Not only they’re returning all the customer money, they were also able to pay somewhere between hundreds of millions and 1.5b (couldn’t find an accurate number) to the lawyers. My impression is that they found a lot of unaccounted money lying around. It’s also not obvious what was the value of Anthropic

MikhailSamin 14 Feb 2024 20:59 UTC
−7 points
3 ∶ 5
in reply to: JWS’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
5- I think the messaging around the protest is deceptive. I.e., it gives people a wrong impression of the world, in a way a part of the community assumed would be better suiting their goals. I think this is deontologically bad and we shouldn’t be even talking about consequences. This is a bad thing to do regardless of how likely and how badly it can backfire. If your calculations say that it’s alright, that the EV for deceiving people is positive, ignore this calculations and maybe read https://www.lesswrong.com/posts/K9ZaZXDnL3SEmYZqB/ends-don-t-justify-means-among-humans.

This is the sort of thing that led to the FTX collapse. This is the sort of decision-making procedure that predictably leads to bad consequences, no matter what our brains calculate.

And in this area, things like that backfiring (e.g., creating an impression among the general public, experts, media, or policymakers that the community isn’t fully candid) can be increasing the chance of a permanent end of life in the lightcone.

4- I’m also interested in mechanisms for early error-correction. This is what my post calls for. I don’t even mention the names of the protest organisers or their orgs, because I care about the community preventing deceptive messaging in the future.

It doesn’t have to be a community-wide mechanism or a central org that checks press-releases; it can be more field-focused orgs and people, it can be practices like fact-checking final versions of public comms and edits to them with outside people who are good at that stuff and have a different background (see comments from @Jason), it can be any of lots of things. I want the community to think about those things and discuss them. I’d be optimistic about people spending time to figure this out and suggesting to the community.

my impression is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work

My impression is that an organiser agreed they haven’t propagated the update, as I suggest in the post, which led to a message that was misleading and this hasn’t been understood or addressed at all until I published the post. I’m not confident this is what happened, though, as looking at the messages now, it seems fairly likely that an organiser understood the impression the messaging would create and decided to keep it that way.

3- I don’t think it’s fair to, while no one of us spent a while thinking about potential mechanisms, suggest those that seem like they wouldn’t seem good/work. Current mechanisms are bad, as they don’t prevent deontologically dubious unilateral actions. I hope there can be better mechanisms.

I think the community as a whole acted deceptively. This seems really bad and the community should probably respond and improve. We don’t want to be the kind of community that misleads people when it suits our goals. I don’t think there’s been a reaction from the community where people tried to figure out how to address the problem. Instead, most of the comments are related to a protest organiser slowly realising what the issue people are talking about was.

2- I notice I’m confused and don’t really get your point. Like, yeah, in global health and development, deceptive messaging is obviously enough unlikely be beneficial for people to not do it and not have much reason to engage with another part of the community having issues around deceptive messaging, so this is not a post that seems relevant to all the community to the extent some parts of the community identify only with these parts and not with the whole. But this is not what you’re saying; and I’m failing to understand your point. If we’re discussing a cause area (x-risk) and deceptive messaging and potentially bad consequences of it, how are other cause area relevant here? I rarely go around shrimp welfare posts, saying that I don’t care about these deep sea shenanigans, and people should donate to MIRI instead.

1- I think better communication could’ve reduced the amount of public drama; but also there’s not a lot of it here, and I’m more concerned with the community not engaging enough with the problems raised in the post.

Thank you for linking the public tweets discussing deontology. If someone believes it’s OK to violate deontological norms if they calculate the consequences to be positive, I want them to directly say that. If someone decided deceptive messaging can be fine and was ok in this case, I want them to directly say that and not be deceptive towards the community as well. We’d then be able to have a discussion about cruxes in strategies and norms and not about what happened.

MikhailSamin 14 Feb 2024 9:06 UTC
3 points
0 ∶ 0
on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
It seems that one of the protest organisers now says that what happened was what I described in the post as “Even the best-intending people are not perfect, often have some inertia, and aren’t able to make sure the messaging they put out isn’t misleading and fully propagate updates”.

I suggest focusing on what mechanisms can be designed to prevent misleading messaging from emerging in the future.

MikhailSamin 14 Feb 2024 8:56 UTC
3 points
0 ∶ 0
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
I wouldn’t care if people knew some number to some approximation and not fully. This is quite different from saying something that’s technically not false but creates a misleading impression you thought was more likely to get people to support your message.

I don’t want to be spending time this way and would be happy if you found someone we’d both be happy with them reading our message exchange and figuring out how deceptive or not was the protest messaging.

MikhailSamin 14 Feb 2024 8:43 UTC
1 point
0 ∶ 0
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans

I thought he was saying mostly that it wasn’t bad to work with the military bc it was cybersecurity, where to me that wasn’t the crux

? It is clearly not what I was telling you. At the end of January, I told you pretty directly that for people who are not aware of the context what you wrote might be misleading, because you omitted crucial details. It’s not about how good or bad what OpenAI are doing is. It’s about people not having important details of the story to judge for themselves.

Mikhail may have felt I was being soldier-y bc I wouldn’t spend more time trying to figure out what he was talking about, but that had more to do with me thinking I had basically understood his point (which I took to be basically that I wasn’t including enough details in the promotional materials so people wouldn’t have a picture he considered accurate enough) and just disagreed with it.

I’m not sure where it’s coming from. I’m not going to publicly share private messages without your permission, but I suggest looking at the messages exchanged around January 31 and double-checking you’re not misleading people here.

MikhailSamin 14 Feb 2024 8:01 UTC
−6 points
0 ∶ 0
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title)

I’m honest in that I consider the community as a whole to have been putting out deceptive messages, which caused the post title; and, separately, I’m honestly saying that while it’s not my main guess for what happened, I can’t rule out the possibility that you understood the messaging could’ve been misleading and still went on with it, accepting that some people can be misled and thinking it’s ok. Things I have heard from you but wouldn’t share publicly don’t inspire confidence.

MikhailSamin 14 Feb 2024 7:51 UTC
1 point
0 ∶ 0
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
(Edit- I no longer believe the above is a true recount of what happened and retract the below.)

Good to see this. I’m sad that you didn’t think that after reading “Even the best-intending people are not perfect, often have some inertia, and aren’t able to make sure the messaging they put out isn’t misleading and fully propagate updates” in the post. As I mentioned before, it could’ve been simply a postmortem.

MikhailSamin 14 Feb 2024 3:28 UTC
−10 points
1 ∶ 2
in reply to: Jason’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
(Edit- I changed the title to better reflect the post contents and what I currently think happened; this comment doesn’t represent my views on the current post title or my current estimates of what happened)

I believe there’s a chance that protest organisers understood their phrasing could potentially cause people to have an impression (OpenAI is working with the military) not informed by details (on cybersecurity and veteran suicide prevention) but kept the phrasing because they thought it suited their goals better.

I don’t expect them to have an active malicious intent (I don’t think they wanted to do something bad). But I think it’s likely enough they were acting deceptively. I consider the chance of them thinking it’s not “misleading” in a bad way and it’s alright to do if it suits the goals to be much higher than what I would be comfortable with. (And people who attended the protest told me that yeah, this is deceptive, after familiarising themselves with the details.)

The title of the post is chosen because I think that regardless of the above, the community (as a whole/as an entity/agent) locally expected to benefit from people having this impression, while knowing it to be a misled impression, and went on with the messaging. Even if individual actors did not take decisions they knew would create impressions they knew were different from what reality is, I think community as a whole can still be deceptive in the sense of having the ability to realise all that and prevent misleading messaging, but not having done so. I think the community should work on strengthening and maybe institutionalising this ability, with the goal of being able to prevent even actively deceptive messaging, where the authors might hope to achieve something good but violate deontology.

MikhailSamin 14 Feb 2024 2:46 UTC
−3 points
3 ∶ 3
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
(Edit- I updated towards them knowing the message is misleading, so this comment no longer correctly represents the level of my uncertainty.)

To be clear, in the post, I’m not saying that you, personally, tried to deceive people. I don’t mention your name or any of the orgs. I’m saying the community as a whole had knowledge of truth but communicated something misleading. My goal with the post is to ask the community to come up with ways of avoiding misleading people. I did not wish for you out your reputation to be involved in any way you wouldn’t want.

But it’s not clear to me what happened, exactly, that led to the misleading messaging being preserved. At the end of January, I shared this with you: “I mean, possibly I don’t know English well and parse this incorrectly, but when I read “OpenAI would be taking the Pentagon as a client” in your tweet, it parses in that context for me as “OpenAI is taking Pentagon as a client to help with offensives”, which is not quite different from what we know”, “ Like, it’s feels different from saying “OpenAI says they’re collaborating with DARPA on cybersecurity. We don’t know whether they’re working with Pentagon on weapons now, but the governance structure wouldn’t stop them from it””, “Like, it doesn’t matter for people aware of the context, but for people without the context, this might be misleading”.

I shared these concerns, thinking that it might be helpful to point out how a message you wrote might be misleading, assuming you were unaware, and not worrying about spending too much time on it.

I’m also confused about the “small ask”- the protest announcement was pretty focused on Pentagon.

(To be clear, I don’t want to be checking anyone’s future comms when this is not my job. I want the community to have mechanisms that don’t involve me. It seemed cheap to make something slightly better a couple of times, and so I talked to a founder of an org organising protests and they fixed some pretty bad claims, sent you these messages, etc., but I dislike the situation where the community doesn’t have mechanisms to systematically avoid mistakes like that.)

MikhailSamin 14 Feb 2024 0:20 UTC
6 points
0 ∶ 0
in reply to: Holly_Elmore’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
Left a comment there, but to repeat:
because I made an error in my original press release
This was, indeed, an honest mistake that you corrected. It is not at all what I’m talking about in the post. I want the community to pay attention to the final messaging being deceptive about the nature of OpenAI’s relationship with the Pentagon.
I am extremely saddened by the emotional impact this has on you. I did not wish that to happen and was surprised and confused by your reaction. Unfortunately, it seems that you still don’t understand the issue I’m pointing at; it is not the “charter” being in the original announcement, it’s the final wording misleading people.