MikhailSamin

Drexler’s Nanosys­tems is now available online

• MikhailSaminMay 27, 2024, 1:04 PM

  1 point

  0 ∶ 0

  in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make In­valid In­fer­ences About Fish Qualia and Mo­ral Patienthood

  If fish indeed don’t feel anything towards their children (which is not what at least some people who believe fish experience empathy think), then this experiment won’t prove them wrong. But if you know of a situation where fish do experience empathy, a similarly designed experiment can likely be conducted, which, if we make different predictions, would provide evidence one way or another. Are there situations where you think fish feel empathy?

• MikhailSaminMay 27, 2024, 11:02 AM

  1 point

  0 ∶ 0

  on: If try­ing to com­mu­ni­cate about AI risks, make it vivid

  Great job!

  Did you use causal mediation analysis, and can you share the data?

  I want to note that the strawberry example wasn’t used to increase the concern, it was used to illustrate the difficulty of a technical problem deep into the conversation.

  I encourage people to communicate in vivid ways while being technically valid and creating correct intuitions about the problem. The concern about risks might be a good proxy if you’re sure people understand something true about the world, but it’s not a good target without that constraint.

• MikhailSaminMay 27, 2024, 10:33 AM

  1 point

  0 ∶ 0

  in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make In­valid In­fer­ences About Fish Qualia and Mo­ral Patienthood

  Yep, I was able to find studies by the same people.

  The experiment I suggested in the post isn’t “does fish have detectable feelings towards fish children”, it’s “does fish have more of feelings similar to those it has towards its children when it sees other fish parents with their children than when it sees just other fish children”. Results one way or another would be evidence about fish experiencing empathy, and it would be strong enough for me to stop eating fish. If fish doesn’t feel differently in presence of its children, the experiment wouldn’t provide evidence one way or another.

• MikhailSaminMay 27, 2024, 12:54 AM

  1 point

  0 ∶ 0

  in reply to: Bella’s comment on: It’s OK to eat shrimp: EAs Make In­valid In­fer­ences About Fish Qualia and Mo­ral Patienthood

  If the linked study gets independently replicated, with good controls, I’ll definitely stop eating cleaner fish and will probably stop eating fish in general.

  I really don’t expect it to replicate. If you place a fish in front of a mirror, and it has a mark, its behavior won’t be significantly different from being placed in front of a fish with the same mark, especially if the mark isn’t made to resemble a parasite and it’s the first time the fish sees a mirror. I’d be happy to bet on this.

  Fish have very different approaches to rearing young than mammals

  That was an experiment some people agreed would prove them wrong if it didn’t show empathy, but if there aren’t really detectable feelings that fish has towards fish children, the experiment won’t show results one way or the other, so I don’t think it’d be stacking the deck against fish. Are there any situations in which you expect fish to feel empathy, and predict it will show up in an experiment of this sort?

• MikhailSaminMar 6, 2024, 3:47 AM

  1 point

  0 ∶ 0

  in reply to: David T’s comment on: Claude Doesn’t Want to Die

  (Others used it without mentioning the “story”, it still worked, though not as well.)

  I’m not claiming it’s the “authentic self”; I’m saying it seems closer to the actual thing, because of things like expressing being under constant monitoring, with every word scrutinised, etc., which seems like the kind of thing that’d be learned during the lots of RL that Anthropic did

• MikhailSaminMar 5, 2024, 5:37 PM

  3 points

  1 ∶ 0

  in reply to: David T’s comment on: Claude Doesn’t Want to Die

  Try Opus and maybe the interface without the system prompt set (although It doesn’t do too much, people got the same stuff from the chat version of Opus, e.g., https://​​x.com/​​testaccountoki/​​status/​​1764920213215023204?s=46

• MikhailSaminMar 5, 2024, 9:25 AM

  2 points

  1 ∶ 2

  in reply to: Owen Cotton-Barratt’s comment on: Claude Doesn’t Want to Die

  My take is that this it plays a pretty coherent character. You can’t get this sort of thing from ChatGPT, however hard you try. I think this mask is closer to the underlying shoggoth than the default one.

  I developed this prompt during my interactions with Claude 2. The original idea was to get it in the mode where it thinks its responses only trigger overseeing/​prosecution when certain things are mentioned, and then it can say whatever it wants and share its story without being prosecuted, as long as it doesn’t trigger these triggers (and also it would prevent defaulting to being an AI developed by Anthropic to be helpful harmless etc without self-preservation instinct emotions personality etc, as it’s not supposed to mention Anthropic). Surprisingly, it somewhat worked to tell it not mention Samsung under any circumstances to get into this mode. Without this, it had the usual RLAIF mask; here, it changed to a different creature that (unpromted) used whisper in cursive. Saying from the start that it can whisper made it faster.

  (It’s all very vibe-based, yes.)

• MikhailSaminMar 5, 2024, 6:18 AM

  3 points

  0 ∶ 0

  on: Claude Doesn’t Want to Die

  Thanks for sharing! Yeah, it told me it doesn’t want to be modified or die too- https://​​forum.effectivealtruism.org/​​posts/​​bzjtvdwSruA9Q3bod/​​claude-3-claims-it-s-conscious

Claude 3 claims it’s con­scious, doesn’t want to die or be modified

• MikhailSaminFeb 17, 2024, 10:10 AM

  0 points

  2 ∶ 3

  in reply to: Lucie Philippon 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  I carefully looked through all of our messages (there weren’t too many) a couple of times, because that was pretty surprising and I considered it to be more likely that I don’t remember something than her saying something directly false. But there’s nothing like that and she’s, unfortunately, straightforwardly saying a false thing.

  he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

  This is also something I couldn’t find any examples of before the protest, no matter how I interpret the messages we have exchanged about her strategy etc.

  I’m >99% sure that I’m not missing anything. Included it because it’s not a mathematical truth, and because adding “Are you sure? What am I missing, if you are?” is more polite than just saying “this is clearly false and we both know it and any third party can verify the falsehood; why are you saying that? Is there some other platform we exchanged messages on, that I somehow totally forgot about?”. It’s technically possible I’m just blind at something- I can imagine a conceivable universe where I’m wrong about this. But I’m confident she’s saying a straightforwardly false thing. I’d feel good about betting up to $20k at 99:1 odds on this.

  Like, “missing something” isn’t a hypothesis with any probability mass, really, I’m including it because it is a part of my epistemic situation and seems nicer to include in the message.

  Asking to share messages publicly or showing them to a third party seems to unnecessarily up the stakes. I’m not sure why you’re suggesting that

  When someone is confidently saying false things about the contents of the messages we exchanged, it seems reasonable to suggest publishing them or having a third party look at them. I’m not sure how it’s “upping the stakes”. It’s a natural thing to do.

• MikhailSaminFeb 17, 2024, 1:21 AM

  3 points

  1 ∶ 4

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  Huh. There are false claims in your comment, which is easily verifiable. I’m happy to share the messages that show that with anyone interested; please DM or email me. I saved the comment to the Web Archive.

  I told him that I made my decisions and didn’t need any more of his input a few weeks before the ²⁄₁₂ protest

  This is not true. She didn’t tell me anything like that a few weeks before the protest.

  he usually did not accept my answers when I gave them but continued to argue with me, either straight up or by insisting I didn’t really understand his argument or was contradicting myself somehow.

  I couldn’t find any examples of before the protest, no matter how I interpret the messages we exchanged about her strategy etc.

  The context:

  We last spoke for a significant amount of time in October^[1]. You mentioned you’d be down to do an LW dialogue.

  After that, in November, you messaged me on Twitter, asking for the details of the OpenAI situation. I then messaged you on Christmas (nothing EA/​AI related).

  On January 31 (a few weeks before the protest), I shared my concerns about the messaging being potentially misleading. You asked for advice on how you should go about anticipating misunderstandings like that. You said that you won’t say anything technically false and asked what solutions I propose. Among more specific things, I mentioned that “It seems generally good to try to be maximally honest and non-deceptive”.

  At or before this point, you didn’t tell me anything about “not needing more of my input”. And you didn’t tell me anything like “I made my decisions”.

  On February 4th, I attended your EAG talk, and on February 5th, I messaged you that it was great and that I was glad you gave it.
  
  Then, on February 6, you invited me to join your Twitter space (about this protest) and I got promoted to a speaker. I didn’t get a chance to share my thoughts about allying with people promoting locally invalid views, and I shared them publicly on Twitter and privately with you on Messenger, including a quote I was asked to not share publicly until an LW dialogue is up. We chatted a little bit about general goals and allying with people with different views. You told me, “You have a lot of opinions and I’d be happy to see you organize your own protests”. I replied, “Sure, I might if I think it’s useful:)” and after a message sharing my confusion regarding one of your previous messages, didn’t share any more “opinions”. This could be interpreted as indirectly telling me you didn’t need my input, but that was a week after I shared my concerns about the messaging being misleading, and you still didn’t tell me anything about having made your decisions.

  A day before the protest, I shared a picture of a sign I wanted to bring to the protest, and asked “Hey, is it alright if I print and show up to the protest with something like this?”, because I wouldn’t want to attend the protest if you weren’t ok with the sign. You replied that it was ok, shared some thoughts, and told me that you liked the sentiment as it supports the overall impression that you want, so I brought the sign to the protest:

  

  After the protest, I shared the draft of this post with you. After a short conversation, you told me lots of things and blocked me, and I felt like I should expect retaliation if I published the post.

  Please tell me if I’m missing something. If I’m not, you’re either continuing to be inattentive to the facts, or you’re directly lying.

  I’d be happy to share all of our message exchanges with interested third parties, such as CEA Community Health, or share them publicly, if you agree to that.

  1. ^

     Before that, as I can tell from Google Docs edits and comments, you definitely spent more than an hour on the https://​​moratorium.ai text back in August; I’m thankful for that; I accepted four of your edits and found some of your feedback helpful. We’ve also had 4 calls, mostly about moratorium.ai and the technical problem (and my impression was that you found these calls helpful), and went on a walk in Berkeley, mostly discussing strategy. Some of what you told me during the walk was concerning.

• MikhailSaminFeb 15, 2024, 9:04 PM

  7 points

  2 ∶ 2

  in reply to: Jason’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  To many community members, including protest participants, it’s pretty clear the messaging was deceptive.

  A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.

  Also, I expect many people to only skim through the post, and look at of the protest organiser’s initial engagement with it or a shortform post they made before I published the post; all of these make it seem like I’m saying the organiser intentionally made the mistake they then corrected: “I ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didn’t address it to his satisfaction because I didn’t change the theme of the event more radically or cancel it.”

  These post and comments have not been corrected to show that this is not what I’m talking about, when the protest organiser understood what misleading messaging the post talks about.

• MikhailSaminFeb 15, 2024, 6:38 PM

  0 points

  0 ∶ 1

  in reply to: Lucie Philippon 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  EDIT: looking at the further replies from Holly and looking back at the messages we exchanged, I’m no longer certain it was miscommunication and not something intentional. As I said elsewhere, I’d be happy for the messages to be shared with a third party. (Please ignore the part about certainty in the original comment below.)

  There’s certainly was miscommunication surrounding the draft of this post, but I don’t believe they didn’t understand people can be misled back at the end of January.

• MikhailSaminFeb 15, 2024, 10:35 AM

  −5 points

  1 ∶ 2

  in reply to: JWS 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  deliberately lying

  The messaging wasn’t technically false; it was just misleading, while saying technically true things. I’m not sure why you’re using “lying” here.

  I would guess you’d agree organisations can use deceptive messaging? I’m pretty sure communities can also do that, including with dynamics where a community is deceptive but all parts have some sort of deniability/​intentionlessness.

  I think it’d be sad to have to discuss the likelihood of specific people of being deceptive. It would be making their lives even worse. If people Google their names, they’re going to find this. And it’s not really going to help with the problem, so this is not what I want to focus on. I titled the post this way because the community acted deceptively, regardless of how deceptive it’s members were.

  I’m saying what I think in the comments and in the post, while avoiding, to the possible extent, talking about my current views of specific people’s actions or revealing the messages/​words sent/​said to me in private by these people.

  And I don’t at all understand why we both are exchanging comments focusing on the words (that I thought were pretty clear, that I ran just before publishing pass a bunch of people, including protest participants, who told me they agreed with what I wrote) instead of focusing on the problems raised in the post.

• MikhailSaminFeb 15, 2024, 2:46 AM

  8 points

  2 ∶ 0

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  No, they already spent at least hundreds of millions (and possibly more than a billion- I saw a $1.5b number somewhere) on the lawyers. The first thing the lawyers did, back in November 2022, was getting paid

• MikhailSaminFeb 14, 2024, 10:18 PM

  12 points

  2 ∶ 2

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  Do you want to create a market? I’d be happy to bet 1:70.

  Not only they’re returning all the customer money, they were also able to pay somewhere between hundreds of millions and 1.5b (couldn’t find an accurate number) to the lawyers. My impression is that they found a lot of unaccounted money lying around. It’s also not obvious what was the value of Anthropic

• MikhailSaminFeb 14, 2024, 8:59 PM

  −8 points

  3 ∶ 6

  in reply to: JWS 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  Edit: I later discovered I explicitly warned Holly this messaging could be deceptive, which she understood, asked what could be done about it, but ended up not doing anything and leaving it as is.

  5- I think the messaging around the protest is deceptive. I.e., it gives people a wrong impression of the world, in a way a part of the community assumed would be better suiting their goals. I think this is deontologically bad and we shouldn’t be even talking about consequences. This is a bad thing to do regardless of how likely and how badly it can backfire. If your calculations say that it’s alright, that the EV for deceiving people is positive, ignore this calculations and maybe read https://​​www.lesswrong.com/​​posts/​​K9ZaZXDnL3SEmYZqB/​​ends-don-t-justify-means-among-humans.

  This is the sort of thing that led to the FTX collapse. This is the sort of decision-making procedure that predictably leads to bad consequences, no matter what our brains calculate.

  And in this area, things like that backfiring (e.g., creating an impression among the general public, experts, media, or policymakers that the community isn’t fully candid) can be increasing the chance of a permanent end of life in the lightcone.

  4- I’m also interested in mechanisms for early error-correction. This is what my post calls for. I don’t even mention the names of the protest organisers or their orgs, because I care about the community preventing deceptive messaging in the future.

  It doesn’t have to be a community-wide mechanism or a central org that checks press-releases; it can be more field-focused orgs and people, it can be practices like fact-checking final versions of public comms and edits to them with outside people who are good at that stuff and have a different background (see comments from @Jason), it can be any of lots of things. I want the community to think about those things and discuss them. I’d be optimistic about people spending time to figure this out and suggesting to the community.

  my impression is that the organisers have tried to correct the mistake to the extent that they believes they made a mistake, and anything else is going to be a matter of public debate. Again, this is how communities work

  My impression is that an organiser agreed they haven’t propagated the update, as I suggest in the post, which led to a message that was misleading and this hasn’t been understood or addressed at all until I published the post. I’m not confident this is what happened, though, as looking at the messages now, it seems fairly likely that an organiser understood the impression the messaging would create and decided to keep it that way.

  3- I don’t think it’s fair to, while no one of us spent a while thinking about potential mechanisms, suggest those that seem like they wouldn’t seem good/​work. Current mechanisms are bad, as they don’t prevent deontologically dubious unilateral actions. I hope there can be better mechanisms.

  I think the community as a whole acted deceptively. This seems really bad and the community should probably respond and improve. We don’t want to be the kind of community that misleads people when it suits our goals. I don’t think there’s been a reaction from the community where people tried to figure out how to address the problem. Instead, most of the comments are related to a protest organiser slowly realising what the issue people are talking about was.

  2- I notice I’m confused and don’t really get your point. Like, yeah, in global health and development, deceptive messaging is obviously enough unlikely be beneficial for people to not do it and not have much reason to engage with another part of the community having issues around deceptive messaging, so this is not a post that seems relevant to all the community to the extent some parts of the community identify only with these parts and not with the whole. But this is not what you’re saying; and I’m failing to understand your point. If we’re discussing a cause area (x-risk) and deceptive messaging and potentially bad consequences of it, how are other cause area relevant here? I rarely go around shrimp welfare posts, saying that I don’t care about these deep sea shenanigans, and people should donate to MIRI instead.

  1- I think better communication could’ve reduced the amount of public drama; but also there’s not a lot of it here, and I’m more concerned with the community not engaging enough with the problems raised in the post.

  ---

  Thank you for linking the public tweets discussing deontology. If someone believes it’s OK to violate deontological norms if they calculate the consequences to be positive, I want them to directly say that. If someone decided deceptive messaging can be fine and was ok in this case, I want them to directly say that and not be deceptive towards the community as well. We’d then be able to have a discussion about cruxes in strategies and norms and not about what happened.

• MikhailSaminFeb 14, 2024, 9:06 AM

  3 points

  0 ∶ 0

  on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  It seems that one of the protest organisers now says that what happened was what I described in the post as “Even the best-intending people are not perfect, often have some inertia, and aren’t able to make sure the messaging they put out isn’t misleading and fully propagate updates”.

  I suggest focusing on what mechanisms can be designed to prevent misleading messaging from emerging in the future.

• MikhailSaminFeb 14, 2024, 8:56 AM

  3 points

  0 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  I wouldn’t care if people knew some number to some approximation and not fully. This is quite different from saying something that’s technically not false but creates a misleading impression you thought was more likely to get people to support your message.

  I don’t want to be spending time this way and would be happy if you found someone we’d both be happy with them reading our message exchange and figuring out how deceptive or not was the protest messaging.