MikhailSamin

[Question] I have thou­sands of copies of HPMOR in Rus­sian. How to use them with the most im­pact?

A tran­script of the TED talk by Eliezer Yudkowsky

FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

NYT is su­ing OpenAI&Microsoft for alleged copy­right in­fringe­ment; some quick thoughts

An EA used de­cep­tive mes­sag­ing to ad­vance their pro­ject; we need mechanisms to avoid de­on­tolog­i­cally du­bi­ous plans

Sav­ing lives near the precipice

• MikhailSamin 24 Aug 2022 22:52 UTC

  17 points

  1 ∶ 0

  on: Open Phil is seek­ing bil­in­gual peo­ple to help trans­late EA/​EA-ad­ja­cent web con­tent into non-English languages

  We’re making translations of a lot of EA content into Russian and have an experience that might be relevant to countries where people mostly can’t speak English.

  We learned that you need introspective people to evaluate or do the translations.

  The best professional translators in our language are mostly hired by large publishing houses and have long-term commitments for many books to come (and the publishing houses aren’t able to lease them for our projects).

  Surprisingly, looking at a translation that looks like a good text in our language but has significant mistakes, most people wouldn’t notice the mistakes. Most people who aren’t the best professional translators don’t actively try to recognize what exactly they just read. When a translation expresses something really different from what the original text conveys, but the words are similar enough, people just don’t notice it.

  Google Translate was better at not making mistakes than 80% of translators that sent us a translated test segment. We ended up hiring two translators from the EA/​LW community.

  A lot of the 80,000 Hours’ articles are highly optimized for conveying a correct understanding, and translation errors might significantly reduce the value of the texts.

• MikhailSamin 4 Jan 2024 22:24 UTC

  15 points

  1 ∶ 0

  in reply to: titotal’s comment on: AI Is Not Software

  The bits of code aren’t inscrutable; the matrices the code makes operations on are.

  The code for Google Meet represents instructions written by humans; the actual image that you see on your screen and the sound that you hear are a result of something else interacting with these instructions. The words from your speaker or headphones are not intended by the Google Meet designers.

  Similarly, the code for GPT-4 represents instructions designed (mostly?) by humans; the actual outputs of GPT-4 are not intended by its designers and depend on the contents of the inscrutable arrays of numbers humans have found.

  We understand that we’re multiplying and taking sums of specific matrices in a specific order; but we have no idea how this is able to lead to the results that we see.

  The important difference here is that normal software implements algorithms designed by humans, run on hardware designed by humans; AI, in contrast, are algorithms blindly designed by an optimisation process designed by humans, run on software designed by humans, but with no understanding of the algorithms implemented by the numbers our optimisation algorithms find.

  It’s like a contrast between CPUs designed by humans and assembly code we don’t understand sent to us by aliens, that we run on CPUs that we do understand

• MikhailSamin 27 Dec 2023 10:29 UTC

  15 points

  7 ∶ 1

  on: Only mam­mals and birds are sen­tient, ac­cord­ing to neu­ro­scien­tist Nick Humphrey’s the­ory of con­scious­ness, re­cently ex­plained in “Sen­tience: The in­ven­tion of con­scious­ness”

  I’m surprised how confident people are in their theories.

  Other mammals and birds exhibiting the same behavior, having much the same neural circuitry, are probably engaging in that behavior for the same reason humans are–because they have internal conscious experiences

  Since self-modelling, having feedback mechanisms for reinforcement learning, and having qualia all feel closely related to us, humans, it’s easy to think that “conscious experience” includes all three, and perceive evidence for some of the three as evidence for others. This seems invalid, unless the author does something like making an argument for how exactly having qualia has been evolutionary useful and what properties it’s connected to and then demonstrating related properties in some animals but not others. Does the author actually do that in the book?

• MikhailSamin 14 Jan 2023 8:27 UTC

  15 points

  4 ∶ 0

  in reply to: Bob Jacobs’s comment on: CEA state­ment on Nick Bostrom’s email

  I don’t think one of the claims, that “Twin studies are flawed in methodology. Twins, even identical twins, simply do not have exactly the same DNA”, is true. As I see, it is not supported by the link and the study.

  The difference of 5.2 out of 6 billion letters that identical twins have on average is not something that makes their DNA distinct enough to make the correlations between being identical tweens or not and having something in common more often to be automatically invalid.

  One of the people involved in the study is cited: “Such genomic differences between identical twins are still very rare. I doubt these differences will have appreciable contribution to phenotypic [or observable] differences in twin studies.”

  Twin studies being something we should be able to rely on seems like a part of the current scientific view, and some EA decisions might take such studies into consideration.

  I think it’s important not to compromise our intellectual integrity even when we debunk foundations for awful and obviously wrong beliefs that are responsible for so much unfairness and suffering that exist in our world and for so many deaths.

  I think if the community uses words that are persuasive but don’t contain actually good evidence, then even if we’re arguing for the truth that’s important and impactful to spread, in the long-term, this might lead to people putting less trust in any of our words arguing for the truth and more people believing something harmful and untrue. And on the internet, there are a lot of words containing bad arguments for the truth because it’s easy for people to be in the mode of finding persuasive arguments, which don’t necessarily have to be actually good evidence.

  I think it’s really important for the EA community to be epistemically honest and talk about the actual reasons we have for believing something, instead of trying to find the most persuasive list of reasons for believing in what we believe in and just copying it without verifying that all the reasons are good and should update people in the claimed direction.

You won’t solve al­ign­ment with­out agent foundations

• MikhailSamin 14 Feb 2024 22:18 UTC

  12 points

  2 ∶ 2

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  Do you want to create a market? I’d be happy to bet 1:70.

  Not only they’re returning all the customer money, they were also able to pay somewhere between hundreds of millions and 1.5b (couldn’t find an accurate number) to the lawyers. My impression is that they found a lot of unaccounted money lying around. It’s also not obvious what was the value of Anthropic

Claude 3 claims it’s con­scious, doesn’t want to die or be modified

• MikhailSamin 15 Feb 2024 2:46 UTC

  8 points

  2 ∶ 0

  in reply to: Nathan Young’s comment on: FTX ex­pects to re­turn all cus­tomer money; claw­backs may go away

  No, they already spent at least hundreds of millions (and possibly more than a billion- I saw a $1.5b number somewhere) on the lawyers. The first thing the lawyers did, back in November 2022, was getting paid

• MikhailSamin 16 Oct 2023 0:46 UTC

  8 points

  0 ∶ 0

  on: Shar­ing In­for­ma­tion About Nonlinear

  Have there been any updates from Nonlinear? Have they written a response?

Please won­der about the hard parts of the al­ign­ment problem

• MikhailSamin 15 Feb 2024 21:04 UTC

  7 points

  2 ∶ 2

  in reply to: Jason’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  To many community members, including protest participants, it’s pretty clear the messaging was deceptive.

  A protest organiser is saying it was the curse of knowledge, but I sent them messages directly pointing out how people will see the messaging. As I mentioned elsewhere in the comments, I want to have a third party look at the messages exchanged between me and the protest organiser, if they agree.

  Also, I expect many people to only skim through the post, and look at of the protest organiser’s initial engagement with it or a shortform post they made before I published the post; all of these make it seem like I’m saying the organiser intentionally made the mistake they then corrected: “I ran a successful protest at [company] yesterday. Before the night was over, Mikhail Samin, who attended the protest, sent me a document to review that accused me of what sounds like a bait and switch and deceptive practices because I made an error in my original press release (which got copied as a description on other materials) and apparently didn’t address it to his satisfaction because I didn’t change the theme of the event more radically or cancel it.”

  These post and comments have not been corrected to show that this is not what I’m talking about, when the protest organiser understood what misleading messaging the post talks about.

• MikhailSamin 14 Feb 2024 0:18 UTC

  7 points

  3 ∶ 1

  in reply to: Holly_Elmore’s comment on: Holly_El­more’s Shortform

  See my post here.

  because I made an error in my original press release

  This was an honest mistake that you corrected. It is not at all what I’m talking about in the post. I want the community to pay attention to the final messaging being deceptive about the nature of OpenAI’s relationship with the Pentagon. The TL;DR of my post:

  Good-hearted EAs lack the mechanisms to not output information that can mislead people. They organised a protest, with the messaging centred on OpenAI changing their documents to “work with the Pentagon”, while OpenAI only collaborates with DARPA on open-source cybersecurity tools and is in talks with the Pentagon about veteran suicide prevention. Many participants of the protest weren’t aware of this; the protest announcement and the press release did not mention this. People were misled into thinking OpenAI is working on military applications of AI. OpenAI still prohibits the use of their services to “harm people, develop weapons, for communications surveillance, or to injure others or destroy property”. If OpenAI wanted to have a contract with the Pentagon to work on something bad, they wouldn’t have needed to change the usage policies of their publicly available services and could’ve simply provided any services through separate agreements. The community should notice a failure mode and implement something that would prevent unilateral decisions with bad consequences or noticeable violations of deontology.

  I am extremely saddened by the emotional impact this has on you. I did not wish that to happen and was surprised and confused by your reaction. Unfortunately, it seems that you still don’t understand the issue I’m pointing at; it is not the “charter” being in the original announcement, it’s the final wording misleading people.

• MikhailSamin 14 Feb 2024 0:20 UTC

  6 points

  0 ∶ 0

  in reply to: Holly_Elmore’s comment on: EA com­mu­nity needs mechanisms to avoid de­cep­tive mes­sag­ing and de­on­tolog­i­cally bad plans

  Left a comment there, but to repeat:

  because I made an error in my original press release

  This was, indeed, an honest mistake that you corrected. It is not at all what I’m talking about in the post. I want the community to pay attention to the final messaging being deceptive about the nature of OpenAI’s relationship with the Pentagon.

  I am extremely saddened by the emotional impact this has on you. I did not wish that to happen and was surprised and confused by your reaction. Unfortunately, it seems that you still don’t understand the issue I’m pointing at; it is not the “charter” being in the original announcement, it’s the final wording misleading people.

• MikhailSamin 6 Jul 2023 7:38 UTC

  6 points

  2 ∶ 4

  on: OpenAI is start­ing a new “Su­per­in­tel­li­gence al­ign­ment” team and they’re hiring

  Our goal is to solve the core technical challenges of superintelligence alignment in four years.

  This is a great goal! I don’t believe they’ve got what it takes to achieve it, though. Safely directing a superintelligent system at solving alignment is an alignment-complete problem. Building a human-level system that does alignment research safely on the first try is possible, running more than one copy of this system at a superhuman speed safely is something no one has any idea how to even approach, and unless this insanity is stopped so we have many more than four years to solve alignment, we’re all dead