Raemon

• Raemon 21 Nov 2025 6:28 UTC

  41 points

  13 ∶ 0

  on: My work­ing group on the best dona­tion opportunities

  I was encouraged by the positive response to my posts: it turned out that many people found them helpful! But that also raised the question: why isn’t anyone else doing this? In a community of people who care a ton about the most effective ways to donate money, why wasn’t anyone else set up to make similarly detailed cost-effectiveness analyses?

  A sort of central paradox of EA as a movement/​community is “you’d think, writing up cost-benefit analysis of donation targets would be like a core community activity”, but, also, there’s big professional orgs evaluating all the charities, and also the a lot of charities feel very fuzzy /​ difficult to evaluate.

  I think it’d be cool if “attempt to make a BOTEC calculation evaluating donation targets” was like the sort of thing people did at EA meetups on-the-regular. (seems more grounding than “spend most of the time recruiting more people to EA”).

[Aceso Un­der Glass] FTX, Golden Geese, and The Wi­dow’s Mite

• Raemon 9 Jul 2025 19:35 UTC

  39 points

  9 ∶ 17

  on: Don’t Eat Honey

  It feels fairly alarming to me that this post didn’t get more pushback here and is so highly upvoted.

  I think it makes a couple interesting points, but then makes extremely crazy sounding claims, taking the Rethink Priorities 7 − 15% numbers at face value, when the arguments for those AFAICT don’t even have particular models behind them. This is a pretty crazy sounding number that needs way better argumentation than “a poll of people said so”, and here it’s just asserted without much commentary at all.

  (In addition to things other people have mentioned here, like the 97% number being very sus, and “why are we assuming they have net negative lives?”, describing “10% as bad as a chicken” as a “conservative assumption” that’s like basically made up. Also, it has some random political potshots that aren’t really affecting the core claim but also seem bad for EA Forum culture)

  This feels like sort of the central example of why EA Vegan Advocacy is not truthseeking, and it’s everyone’s problem needed to get written. (disclaimer: I am close with the author of that post)

  (By contrast it has negative karma on LessWrong. I have weak disagreement with Oliver there about whether it should be more like −9 karma or more like “-2 to 10”, but it was at 85 karma when I found it here before me and a couple people strong downvoted and that seems like EA forum basically has no filter for poorly argued claims)

• Raemon 14 Sep 2024 1:48 UTC

  10 points

  1 ∶ 0

  on: How to help cru­cial AI safety leg­is­la­tion pass with 10 min­utes of effort

  I have written a letter

• Raemon 11 Sep 2024 21:15 UTC

  3 points

  0 ∶ 0

  on: How to help cru­cial AI safety leg­is­la­tion pass with 10 min­utes of effort

  I recall previously hearing there might be a final round of potential amendments in response to things Gavin Newsom requests. Was/​is that accurate?

• Raemon 23 Aug 2024 5:41 UTC

  2 points

  0 ∶ 0

  in reply to: Greg S’s comment on: Re­cur­sive Mid­dle Man­ager Hell

  (several years late, whoops!)

  Yeah, my intent here was more “be careful deciding to scale your company to the point you need a lot of middle managers, if you have a nuanced goal”, rather than “try to scale your company without middle managers.”

• Raemon 13 Aug 2024 18:19 UTC

  3 points

  0 ∶ 0

  in reply to: Neel Nanda’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  In the context of an EA jobs list it seems like both are pretty bad. (there’s the “job list” part, and the “EA” part)

• Raemon 8 Aug 2024 18:25 UTC

  14 points

  3 ∶ 0

  in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Yeah, this does seem like an improvement. I appreciate you thinking about it and making some updates.

• Raemon 2 Aug 2024 15:53 UTC

  2 points

  0 ∶ 0

  in reply to: Zach Stein-Perlman’s comment on: Cor­po­rate AI Labs’ Odd Role in Their Own Governance

  Can you say a bit more about:

  and (2) worse in private than in public.

  ?

Op­ti­mistic As­sump­tions, Longterm Plan­ning, and “Cope”

Fluent, Cruxy Predictions

• Raemon 6 Jul 2024 1:06 UTC

  2 points

  0 ∶ 0

  in reply to: Habryka [Deactivated]’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Mmm, nod. I will look into the actual history here more, but, sounds plausible. (edited the previous comment a bit for now)

• Raemon 5 Jul 2024 23:51 UTC

  9 points

  1 ∶ 0

  in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Following up my other comment:

  To try to be a bit more helpful rather than just complaining and arguing: when I model your current worldview, and try to imagine a disclaimer that helps a bit more with my concerns but seems like it might work for you given your current views, here’s a stab. Changes bolded.

  OpenAI is a frontier AI research and product company, with teams working on alignment, policy, and security. We recommend specific opportunities at OpenAI that we think may be high impact. We recommend applicants pay attention to the details of individual roles at OpenAI, and form their own judgment about whether the role is net positive. We do not necessarily recommend working at other positions at OpenAI

  You can read considerations around working at a frontier AI company in our career review on the topic.

  (it’s not my main crux, by “frontier” felt both like a more up-to-date term for what OpenAI does, and also feels more specifically like it’s making a claim about the product than generally awarding status to the company the way “leading” does)

• Raemon 5 Jul 2024 19:54 UTC

  35 points

  9 ∶ 0

  in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Thanks.

  Fwiw while writing the above, I did also think “hmm, I should also have some cruxes for ‘what would update me towards ‘these jobs are more real than I currently think.’” I’m mulling that over and will write up some thoughts soon.

  It sounds like you basically trust their statements about their roles. I appreciate you stating your position clearly, but, I do think this position doesn’t make sense:

  • we already have evidence of them failing to uphold commitments they’ve made in clear cut ways. (i.e. I’d count their superalignment compute promises as basically a straightforward lie, and if not a “lie”, it at least clearly demonstrates that their written words don’t count for much. This seems straightforwardly relevant to the specific topic of “what does a given job at OpenAI entail?”, in addition to being evidence about their overall relationship with existential safety)

  • we’ve similarly seen OpenAI change it’s stated policies, such as removing restrictions on military use. Or, initially being a nonprofit and converting into “for-profit-managed by non-profit” (where the “managed by nonprofit board” part turned out to be pretty ineffectual) (not sure if I endorse this, mulling over Habryka’s comment)

  Surely, this at at least updates you downward on how trustworthy their statements are? How many times do they have to “say things that turned out not to be true” before you stop taking them at face value? And why is that “more times than they have already?”.

  Separate from straightforward lies, and/​or altering of policy to the point where any statements they make seem very unreliable, there is plenty of degrees of freedom of “what counts as alignment.” They are already defining alignment in a way that is pretty much synonymous with short-term capabilities. I think the plan of “iterate on ‘alignment’ with nearterm systems as best you can to learn and prepare” is not necessarily a crazy plan. There are people I respect who endorse it, who previously defended it as an OpenAI approach, although notably most of those people have now left OpenAI (sometimes still working on similar plans at other orgs).

  But, it’s very hard to tell the difference from the outside between:

  • “iterating on nearterm systems, contributing to AI race dynamics in the process, in a way that has a decent chance of teaching you skills that will be relevant for aligning superintelligences”

  • “iterating on nearterm systems, in a way that you think/​hope will teach you skills for navigating superintelligence… but, you’re wrong about how much you’re learning, and whether it’s net positive”

  • “iterating on nearterm systems, and calling it alignment because it makes for better PR, but not even really believing that it’s particularly necessary to navigate superintelligence.

  When recommending jobs for organizations that are potentially causing great harm, I think 80k has a responsibility to actually form good opinions on whether the job makes sense, independent on what the organization says it’s about.

  You don’t just need to model whether OpenAI is intentionally lying, you also need to model whether they are phrasing things ambiguously, and you need to model whether they are self-decelving about whether these roles are legitimate alignment work, or valuable enough work to outweigh the risks. And, you need to model that they might just be wrong and incompetent at longterm alignment development (or: “insufficiently competent to outweigh risks and downsides”), even if their heart were in the right place.

  I am very worried that this isn’t already something you have explicit models about.

• Raemon 5 Jul 2024 19:46 UTC

  2 points

  1 ∶ 0

  in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Thanks. This still seems pretty insufficient to me, but, it’s at least an improvement and I appreciate you making some changes here.

• Raemon 5 Jul 2024 17:57 UTC

  2 points

  0 ∶ 0

  in reply to: Habryka [Deactivated]’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Yeah same. (although, this focuses entirely on their harm as an AI organization, and not manipulative practices)

  I think it leaves the question “what actually is the above-the-fold-summary” (which’d be some kind of short tag).

• Raemon 4 Jul 2024 19:42 UTC

  5 points

  5 ∶ 2

  in reply to: JackM’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  I think EAs vary wildly. I think most EAs do not have those skills – I think it is a very difficult skill. Merely caring about the world is not enough.

  I think most EAs do not, by default, prioritize epistemics that highly, unless they came in through the rationalist scene, and even then, I think holding onto your epistemics while navigating social pressure is a very difficult skill that even rationalists who specialize in it tend to fail at. (Getting into details here is tricky because it involves judgment calls about individuals, in social situations that are selected for being murky and controversial, but, no, I do not think the median EA or even the 99th percentile EA is going to competent enough at this for it to be worthwhile for them to join OpenAI. I think ~99.5th percentile is the point where it seems even worth talking about, and I don’t think those people get most of their job leads through the job board).

• Raemon 4 Jul 2024 19:21 UTC

  3 points

  0 ∶ 0

  in reply to: JackM’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  Surely this isn’t the typical EA though?

  I think job ads in particular are a filter for “being more typical.”

  I expect the people who have a chance of doing a good job to be well connected to previous people who worked at OpenAI, with some experience under their belt navigating organizational social scenes while holding onto their own epistemics. I expect such a person to basically not need to see the job ad.

• Raemon 4 Jul 2024 19:12 UTC

  12 points

  0 ∶ 0

  on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  I do want to acknowledge:

  I refer to Jan Leike’s and Daniel Kokotajlo’s comments about why the left, and reference other people leaving the company.

  I do think this is important evidence.

  I want to acknowledge I wouldn’t actually bet that Jan and Daniel would endorse everyone else leaving OpenAI, and only weakly bet that they’d endorse not leaving up the current 80k-ads as written.

  I am grateful to them for having spoken up publicly, but I know that a reason people hesitate to speak publicly about this sort of thing is that it’s easier for soundbyte words to get taken and runaway by people arguing for positions stronger than you endorse, and I don’t want them to regret that.

  I know at least one person who has less negative (but mixed) feelings who left OpenAI for somewhat different reasons, and another couple people who still work at OpenAI I respect in at least some domains.

  (I haven’t chatted with either of them about this recently)

• Raemon 4 Jul 2024 19:07 UTC

  6 points

  0 ∶ 0

  in reply to: Conor Barnes 🔶’s comment on: 80,000 hours should re­move OpenAI from the Job Board (and similar EA orgs should do similarly)

  I have slightly complex thoughts about the “is 80k endorsing OpenAI?” question.

  I’m generally on the side of “let people make individual statements without treating it as a blanket endorsement.”

  In practice, I think the job postings will be read as an endorsement by many (most?) people. But I think the overall policy of “social-pressure people to stop making statements that could be read as endorsements” is net harmful.

  I think you should at least be acknowledging the implication-of-endorsement as a cost you are paying.

  I’m a bit confused about how to think about it here, because I do think listing people on the job site, with the sorts of phrasing you use, feels more like some sort of standard corporate political move than a purely epistemic move.

  I do want to distinguish the question of “how does this job-ad funnel social status around?” from “does this job-ad communicate clearly?”. I think it’s still bad to force people only speak words that can’t be inaccurately read into, but, I think this is an important enough area to put extra effort in.

  An accurate job posting, IMO, would say “OpenAI-in-particular has demonstrated that they do not follow through on safety promises, and we’ve seen people leave due to not feeling effectual.”

  I think you maybe both disagree with that object level fact (if so, I think you are wrong, and this is important), as well as, well, that’d be a hell of a weird job ad. Part of why I am arguing here is I think it looks, from the outside, like 80k is playing a slightly confused mix of relating to orgs politically and making epistemic recommendations.

  I kind of expect at this point you to leave the job ad up, and maybe change the disclaimer slightly in a way that is leaves some sort of plausibly-deniable veneer.