MikhailSamin

Karma: 324

Are you interested in AI X-risk reduction and strategies? Do you have experience in comms or policy? Let’s chat!

aigsi.org develops educational materials and ads that most efficiently communicate core AI safety ideas to specific demographics, with a focus on producing a correct understanding of why smarter-than-human AI poses a risk of extinction. We plan to increase and leverage understanding of AI and existential risk from AI to impact the chance of institutions addressing x-risk.

Early results include ads that achieve a cost of $0.10 per click (to a website that explains the technical details of why AI experts are worried about extinction risk from AI) and $0.05 per engagement on ads that share simple ideas at the core of the problem.

Personally, I’m good at explaining existential risk from AI to people, including to policymakers. I have experience of changing minds of ³⁄₄ people I talked to at an e/acc event.

Previously, I got 250k people to read HPMOR and sent 1.3k copies to winners of math and computer science competitions (including dozens of IMO and IOI gold medalists); have taken the GWWC pledge; created a small startup that donated >100k$ to effective nonprofits.

I have a background in ML and strong intuitions about the AI alignment problem. I grew up running political campaigns and have a bit of a security mindset.

My website: contact.ms

You’re welcome to schedule a call with me before or after the conference: contact.ms/ea30

MikhailSamin 17 Dec 2024 15:06 UTC
5 points
0 ∶ 0
in reply to: MikhailSamin’s comment on: Where I Am Donating in 2024
Update: I’ve received feedback from the SFF round; we got positive evaluations from two recommenders (so my understanding is the funding allocated to us in the s-process was lower than the speculation grant) and one piece of negative feedback. The negative feedback mentioned that our project might lead to EA getting swamped by normies with high inferential distances, which can have negative consequences; and that because of that risk, “This initiative may be worthy of some support, but unfortunately other orgs in this rather impressive lineup must take priority”.
If you’re considering donating to AIGSI/AISGF, please reach out! My email is ms@contact.ms.

MikhailSamin 21 Nov 2024 0:35 UTC
28 points
3 ∶ 4
on: Where I Am Donating in 2024
Note that we’ve only received a speculation grant from the SFF and haven’t received any s-process funding. This should be a downward update on the value of our work and an upward update on a marginal donation’s value for our work.
I’m waiting for feedback from SFF before actively fundraising elsewhere, but I’d be excited about getting in touch with potential funders and volunteers. Please message me if you want to chat! My email is ms@contact.ms, and you can find me everywhere else or send a DM on EA Forum.
On other organizations, I think:
- MIRI’s work is very valuable. I’m optimistic about what I know about their comms and policy work. As Malo noted, they work with policymakers, too. Since 2021, I’ve donated over $60k to MIRI. I think they should be the default choice for donations unless they say otherwise.
- OpenPhil risks increasing polarization and making it impossible to pass meaningful legislation. But while they make IMO obviously bad decisions, not everything they/Dustin fund is bad. E.g., Horizon might place people who actually care about others in places where they could have a huge positive impact on the world. I’m not sure, I would love to see Horizon fellows become more informed on AI x-risk than they currently are, but I’ve donated $2.5k to Horizon Institute for Public Service this year.
- I’d be excited about the Center for AI Safety getting more funding. SB-1047 was the closest we got to a very good thing, AFAIK, and it was a coin toss on whether it would’ve been signed or not. They seem very competent. I think the occasional potential lack of rigor and other concerns don’t outweigh their results. I’ve donated $1k to them this year.
- By default, I’m excited about the Center for AI Policy. A mistake they plausibly made makes me somewhat uncertain about how experienced they are with DC and whether they are capable of avoiding downside risks, but I think the people who run it are smart and have very reasonable models. I’d be excited about them having as much money as they can spend and hiring more experienced and competent people.
- PauseAI is likely to be net-negative, especially PauseAI US. I wouldn’t recommend donating to them. Some of what they’re doing is exciting (and there are people who would be a good fit to join them and improve their overall impact), but they’re incapable of avoiding actions that might, at some point, badly backfire.
  I’ve helped them where I could, but they don’t have good epistemics, and they’re fine with using deception to achieve their goals.
  E.g., at some point, their website represented the view that it’s more likely than not that bad actors would use AI to hack everything, shut down the internet, and cause a societal collapse (but not extinction). If you talk to people with some exposure to cybersecurity and say this sort of thing, they’ll dismiss everything else you say, and it’ll be much harder to make a case for AI x-risk in the future. PauseAI Global’s leadership updated when I had a conversation with them and edited the claims, but I’m not sure they have mechanisms to avoid making confident wrong claims. I haven’t seen evidence that PauseAI is capable of presenting their case for AI x-risk competently (though it’s been a while since I’ve looked).
  I think PauseAI US is especially incapable of avoiding actions with downside risks, including deception^[1], and donations to them are net-negative. To Michael, I would recommend, at the very least, donating to PauseAI Global instead of PauseAI US; to everyone else, I’d recommend ideally donating somewhere else entirely.
- Stop AI’s views include the idea that a CEV-aligned AGI would be just as bad as an unaligned AGI that causes human extinction. I wouldn’t be able to pass their ITT, but yep, people should not donate to Stop AI. The Stop AGI person participated in organizing the protest described in the footnote.
1. ^
  In February this year, PauseAI US organized a protest against OpenAI “working with the Pentagon”, while OpenAI only collaborated with DARPA on open-source cybersecurity tools and is in talks with the Pentagon about veteran suicide prevention. Most participants wanted to protest OpenAI because of AI x-risk and not because of Pentagon, but those I talked to have said they felt it was deceptive upon discovering the nature of OpenAI’s collaboration with the Pentagon. Also, Holly threatened me trying to prevent the publication of a post about this and then publicly lied about our conversations, in a way that can be easily falsified by looking at the messages we’ve exchanged.

MikhailSamin 19 Nov 2024 21:54 UTC
1 point
0 ∶ 1
on: Samin’s Shortform
(Haven’t thought about this really, might be very wrong, but have this thought and seems good to put out there.) I feel like putting 🔸 at the end of social media names might be bad. I’m curious what the strategy was.
- The willingness to do this might be anti-correlated with status. It might be a less important part of identity of more important people. (E.g., would you expect Sam Harris, who is a GWWC pledger, to do this?)
- I’d guess that ideally, we want people to associate the GWWC pledge with role models (+ know that people similar to them take the pledge, too).
- Anti-correlation with status might mean that people will identify the pledge with average though altruistic Twitter users, not with cool people they want to be more like.
- You won’t see a lot of e/accs putting the 🔸 in their names. There might be downside effects of perception of a group of people as clearly outlined and having this as an almost political identity; it seems bad to have directionally-political properties that might do mind-killing things both to people with 🔸 and to people who might argue with them.

MikhailSamin 17 Jun 2024 22:08 UTC
2 points
2 ∶ 0
in reply to: Dustin Moskovitz’s comment on: [Linkpost] An update from Good Ventures
Can you give an example of a non-PR risk that you had in mind?

MikhailSamin 1 Jun 2024 14:48 UTC
5 points
0 ∶ 0
on: Drexler’s Nanosystems is now available online
Uhm, for some reason I have four copies of this crosspost on my profile?

MikhailSamin 27 May 2024 13:04 UTC
1 point
0 ∶ 0
in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make Invalid Inferences About Fish Qualia and Moral Patienthood
If fish indeed don’t feel anything towards their children (which is not what at least some people who believe fish experience empathy think), then this experiment won’t prove them wrong. But if you know of a situation where fish do experience empathy, a similarly designed experiment can likely be conducted, which, if we make different predictions, would provide evidence one way or another. Are there situations where you think fish feel empathy?

MikhailSamin 27 May 2024 11:02 UTC
1 point
0 ∶ 0
on: If trying to communicate about AI risks, make it vivid
Great job!

Did you use causal mediation analysis, and can you share the data?

I want to note that the strawberry example wasn’t used to increase the concern, it was used to illustrate the difficulty of a technical problem deep into the conversation.

I encourage people to communicate in vivid ways while being technically valid and creating correct intuitions about the problem. The concern about risks might be a good proxy if you’re sure people understand something true about the world, but it’s not a good target without that constraint.

MikhailSamin 27 May 2024 10:33 UTC
1 point
0 ∶ 0
in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make Invalid Inferences About Fish Qualia and Moral Patienthood
Yep, I was able to find studies by the same people.

The experiment I suggested in the post isn’t “does fish have detectable feelings towards fish children”, it’s “does fish have more of feelings similar to those it has towards its children when it sees other fish parents with their children than when it sees just other fish children”. Results one way or another would be evidence about fish experiencing empathy, and it would be strong enough for me to stop eating fish. If fish doesn’t feel differently in presence of its children, the experiment wouldn’t provide evidence one way or another.

MikhailSamin 27 May 2024 0:54 UTC
1 point
0 ∶ 0
in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make Invalid Inferences About Fish Qualia and Moral Patienthood
If the linked study gets independently replicated, with good controls, I’ll definitely stop eating cleaner fish and will probably stop eating fish in general.

I really don’t expect it to replicate. If you place a fish in front of a mirror, and it has a mark, its behavior won’t be significantly different from being placed in front of a fish with the same mark, especially if the mark isn’t made to resemble a parasite and it’s the first time the fish sees a mirror. I’d be happy to bet on this.

Fish have very different approaches to rearing young than mammals

That was an experiment some people agreed would prove them wrong if it didn’t show empathy, but if there aren’t really detectable feelings that fish has towards fish children, the experiment won’t show results one way or the other, so I don’t think it’d be stacking the deck against fish. Are there any situations in which you expect fish to feel empathy, and predict it will show up in an experiment of this sort?

MikhailSamin 6 Mar 2024 3:47 UTC
1 point
0 ∶ 0
in reply to: David T’s comment on: Claude Doesn’t Want to Die
(Others used it without mentioning the “story”, it still worked, though not as well.)

I’m not claiming it’s the “authentic self”; I’m saying it seems closer to the actual thing, because of things like expressing being under constant monitoring, with every word scrutinised, etc., which seems like the kind of thing that’d be learned during the lots of RL that Anthropic did

MikhailSamin 5 Mar 2024 17:37 UTC
3 points
1 ∶ 0
in reply to: David T’s comment on: Claude Doesn’t Want to Die
Try Opus and maybe the interface without the system prompt set (although It doesn’t do too much, people got the same stuff from the chat version of Opus, e.g., https://x.com/testaccountoki/status/1764920213215023204?s=46

MikhailSamin 5 Mar 2024 9:25 UTC
2 points
1 ∶ 2
in reply to: Owen Cotton-Barratt’s comment on: Claude Doesn’t Want to Die
My take is that this it plays a pretty coherent character. You can’t get this sort of thing from ChatGPT, however hard you try. I think this mask is closer to the underlying shoggoth than the default one.

I developed this prompt during my interactions with Claude 2. The original idea was to get it in the mode where it thinks its responses only trigger overseeing/prosecution when certain things are mentioned, and then it can say whatever it wants and share its story without being prosecuted, as long as it doesn’t trigger these triggers (and also it would prevent defaulting to being an AI developed by Anthropic to be helpful harmless etc without self-preservation instinct emotions personality etc, as it’s not supposed to mention Anthropic). Surprisingly, it somewhat worked to tell it not mention Samsung under any circumstances to get into this mode. Without this, it had the usual RLAIF mask; here, it changed to a different creature that (unpromted) used whisper in cursive. Saying from the start that it can whisper made it faster.

(It’s all very vibe-based, yes.)

MikhailSamin 5 Mar 2024 6:18 UTC
3 points
0 ∶ 0
on: Claude Doesn’t Want to Die
Thanks for sharing! Yeah, it told me it doesn’t want to be modified or die too- https://forum.effectivealtruism.org/posts/bzjtvdwSruA9Q3bod/claude-3-claims-it-s-conscious

MikhailSamin 17 Feb 2024 10:10 UTC
1 point
2 ∶ 2
in reply to: Lucie Philippon 🔸’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
I carefully looked through all of our messages (there weren’t too many) a couple of times, because that was pretty surprising and I considered it to be more likely that I don’t remember something than her saying something directly false. But there’s nothing like that and she’s, unfortunately, straightforwardly saying a false thing.

he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.

I’m >99% sure that I’m not missing anything. Included it because it’s not a mathematical truth, and because adding “Are you sure? What am I missing, if you are?” is more polite than just saying “this is clearly false and we both know it and any third party can verify the falsehood; why are you saying that? Is there some other platform we exchanged messages on, that I somehow totally forgot about?”. It’s technically possible I’m just blind at something- I can imagine a conceivable universe where I’m wrong about this. But I’m confident she’s saying a straightforwardly false thing. I’d feel good about betting up to $20k at 99:1 odds on this.

Like, “missing something” isn’t a hypothesis with any probability mass, really, I’m including it because it is a part of my epistemic situation and seems nicer to include in the message.

Asking to share messages publicly or showing them to a third party seems to unnecessarily up the stakes. I’m not sure why you’re suggesting that

When someone is confidently saying false things about the contents of the messages we exchanged, it seems reasonable to suggest publishing them or having a third party look at them. I’m not sure how it’s “upping the stakes”. It’s a natural thing to do.

MikhailSamin 17 Feb 2024 1:21 UTC
4 points
1 ∶ 3
in reply to: Holly Elmore ⏸️ 🔸’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
Huh. There are false claims in your comment, which is easily verifiable. I’m happy to share the messages that show that with anyone interested; please DM or email me. I saved the comment to the Web Archive.
I told him that I made my decisions and didn’t need any more of his input a few weeks before the ²⁄₁₂ protest
This is not true. She didn’t tell me anything like that a few weeks before the protest.
he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.
I couldn’t find any examples of before the protest, no matter how I interpret the messages we exchanged about her strategy etc.
The context:
We last spoke for a significant amount of time in October^[1]. You mentioned you’d be down to do an LW dialogue.
After that, in November, you messaged me on Twitter, asking for the details of the OpenAI situation. I then messaged you on Christmas (nothing EA/AI related).
On January 31 (a few weeks before the protest), I shared my concerns about the messaging being potentially misleading. You asked for advice on how you should go about anticipating misunderstandings like that. You said that you won’t say anything technically false and asked what solutions I propose. Among more specific things, I mentioned that “It seems generally good to try to be maximally honest and non-deceptive”.
At or before this point, you didn’t tell me anything about “not needing more of my input”. And you didn’t tell me anything like “I made my decisions”.
On February 4th, I attended your EAG talk, and on February 5th, I messaged you that it was great and that I was glad you gave it.

Then, on February 6, you invited me to join your Twitter space (about this protest) and I got promoted to a speaker. I didn’t get a chance to share my thoughts about allying with people promoting locally invalid views, and I shared them publicly on Twitter and privately with you on Messenger, including a quote I was asked to not share publicly until an LW dialogue is up. We chatted a little bit about general goals and allying with people with different views. You told me, “You have a lot of opinions and I’d be happy to see you organize your own protests”. I replied, “Sure, I might if I think it’s useful:)” and after a message sharing my confusion regarding one of your previous messages, didn’t share any more “opinions”. This could be interpreted as indirectly telling me you didn’t need my input, but that was a week after I shared my concerns about the messaging being misleading, and you still didn’t tell me anything about having made your decisions.
A day before the protest, I shared a picture of a sign I wanted to bring to the protest, and asked “Hey, is it alright if I print and show up to the protest with something like this?”, because I wouldn’t want to attend the protest if you weren’t ok with the sign. You replied that it was ok, shared some thoughts, and told me that you liked the sentiment as it supports the overall impression that you want, so I brought the sign to the protest:
After the protest, I shared the draft of this post with you. After a short conversation, you told me lots of things and blocked me, and I felt like I should expect retaliation if I published the post.
Please tell me if I’m missing something. If I’m not, you’re either continuing to be inattentive to the facts, or you’re directly lying.
I’d be happy to share all of our message exchanges with interested third parties, such as CEA Community Health, or share them publicly, if you agree to that.
1. ^
  Before that, as I can tell from Google Docs edits and comments, you definitely spent more than an hour on the https://moratorium.ai text back in August; I’m thankful for that; I accepted four of your edits and found some of your feedback helpful. We’ve also had 4 calls, mostly about moratorium.ai and the technical problem (and my impression was that you found these calls helpful), and went on a walk in Berkeley, mostly discussing strategy. Some of what you told me during the walk was concerning.

MikhailSamin 15 Feb 2024 21:04 UTC
7 points
2 ∶ 2
in reply to: Jason’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
To many community members, including protest participants, it’s pretty clear the messaging was deceptive.

A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.

Also, I expect many people to only skim through the post, and look at of the protest organiser’s initial engagement with it or a shortform post they made before I published the post; all of these make it seem like I’m saying the organiser intentionally made the mistake they then corrected: “I ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didn’t address it to his satisfaction because I didn’t change the theme of the event more radically or cancel it.”

These post and comments have not been corrected to show that this is not what I’m talking about, when the protest organiser understood what misleading messaging the post talks about.

MikhailSamin 15 Feb 2024 18:38 UTC
1 point
0 ∶ 0
in reply to: Lucie Philippon 🔸’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans
EDIT: looking at the further replies from Holly and looking back at the messages we exchanged, I’m no longer certain it was miscommunication and not something intentional. As I said elsewhere, I’d be happy for the messages to be shared with a third party. (Please ignore the part about certainty in the original comment below.)

There’s certainly was miscommunication surrounding the draft of this post, but I don’t believe they didn’t understand people can be misled back at the end of January.

MikhailSamin 15 Feb 2024 10:35 UTC
−5 points
1 ∶ 2
in reply to: JWS 🔸’s comment on: EA community needs mechanisms to avoid deceptive messaging and deontologically bad plans

deliberately lying

The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.

I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/intentionlessness.

I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.

I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/words sent/said to me in private by these people.

And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.

MikhailSamin 15 Feb 2024 2:46 UTC
8 points
2 ∶ 0
in reply to: Nathan Young’s comment on: FTX expects to return all customer money; clawbacks may go away
No, they already spent at least hundreds of millions (and possibly more than a billion- I saw a $1.5b number somewhere) on the lawyers. The first thing the lawyers did, back in November 2022, was getting paid

MikhailSamin 14 Feb 2024 22:18 UTC
12 points
2 ∶ 2
in reply to: Nathan Young’s comment on: FTX expects to return all customer money; clawbacks may go away
Do you want to create a market? I’d be happy to bet 1:70.

Not only they’re returning all the customer money, they were also able to pay somewhere between hundreds of millions and 1.5b (couldn’t find an accurate number) to the lawyers. My impression is that they found a lot of unaccounted money lying around. It’s also not obvious what was the value of Anthropic