Ben Millwood🔸

Karma: 4,782

Ben Millwood🔸 21 May 2026 21:10 UTC
5 points
0 ∶ 0
on: Why are longtermists so much less focused on human extinction these days?
These might be kinda similar to things that others have already said, but:

My personal journey was encountering extinction risks first, worrying about those, and then over time thinking in more detail about threat models and consequently broadening the list of things I worried about. I’ve been assuming that community discourse evolved in the same way: initially based on relatively simple ideas (e.g. omnipotent superintelligence, everyone dies) and then adding more detail and precision and subtlety as people developed those more, which naturally increases the number of possible pathways and scenarios. But it’s possible that all the new pathways I discovered were only new to me, and therefore my path doesn’t track the community path. I don’t know.

On the object level, I don’t super buy the “extinction is positive lock-in but not much else is”. Similar to what Vasco said, you only believe that extinction has a massive impact on the long-term future if you believe that extinction risk is high now but will drop to being permanently extremely low later on. Otherwise you’re just delaying extinction rather than preventing it. This isn’t an obvious belief! Most who believe it appeal to something like space colonisation, where we spread in such a way that it’s no longer easy for effects of any kind to spread across all sentient life rapidly. But if space colonisation works to prevent extinction (which itself is not obvious!) then maybe it also locks in other things in the same way. You can think of not only the human race as having their survival at stake here, but also our ideas, social structures, etc. -- arguably, space colonisation gives these things the same shot at immortality it does us.

Having written this out, I think it’s fair enough if you still think that “locking in non-extinction” is likely to happen, but locking in anything else isn’t. It’s reasonable to believe that extinction is special. But I hope that gives some intuition for why someone might think that extinction and lock-in are comparable risks.

Ben Millwood🔸 21 May 2026 21:05 UTC
7 points
3 ∶ 1
in reply to: Nathan Young’s comment on: EA sexual harassment: Where’s the evidence we’re worse than baseline?
“shockingly terrible”, no, that’s still not a comparison.

“far worse than anything I’ve ever seen at any organisation I’ve ever been in”—this one is a comparison! (In my defence I hadn’t seen titotal’s comment when I made mine.)

But here it doesn’t make a lot of sense to me to respond to that comparison with “where’s your evidence?” or “what about Durham?”. It’s clear where the comparison is coming from, and it’s clear that it’s not about base rates, it’s about one specific org and a specific thing they did, and it’s specific about what standard they’re held to. Are you saying that it’s the wrong standard?

Ben Millwood🔸 21 May 2026 20:46 UTC
4 points
1 ∶ 0
in reply to: Nathan Young’s comment on: EA sexual harassment: Where’s the evidence we’re worse than baseline?
I think the third one most clearly. Doesn’t suggest that other communities successfully take sexual harassment seriously, to my eyes.

Ben Millwood🔸 21 May 2026 20:37 UTC
23 points
5 ∶ 0
in reply to: Nathan Young’s comment on: EA sexual harassment: Where’s the evidence we’re worse than baseline?
I think you’re reading some of them right, and many of them wrong, because you seem to continue to be equating “EA’s performance on this issue makes me sad” and “EA’s performance on this issue is worse than peers”. Some people believe both for sure! But you keep including people saying the first thing as if they’re saying the second. “Currently about half of the comments disagreeing here seem to espouse the view that the community is bad.”—again not distinguishing between “bad” and “worse”.

Honestly my guess would be that most people don’t have a clear considered belief on the comparison. They see bad behaviour and they object. It’s not obvious to me why they would feel the need for a belief on the comparison. Whatever it is, the bad behaviour is still objectionable.

I hear you as pushing in a direction of “maybe we can’t do anything about it, because no-one around us has succeeded in doing better”. And, well, maybe! But I think this is a weak heuristic as compared with thinking directly about whether we should have stronger codes of conduct at EA conferences, or whether we need to develop more training resources or other support for orgs that are too small to have a proper HR function, or what stopped CEA from acting on what seems like clear evidence of inappropriate behaviour.

Ben Millwood🔸 21 May 2026 20:17 UTC
5 points
1 ∶ 0
in reply to: Dave Cortright 🔸’s comment on: Why are longtermists so much less focused on human extinction these days?
I think of this situation as analogous to waking up in the cabin of a truck that’s careening down a major road. Of course you’re going to grab the wheel, just because no-one else has done so yet.

Ben Millwood🔸 21 May 2026 19:56 UTC
17 points
6 ∶ 2
on: EA sexual harassment: Where’s the evidence we’re worse than baseline?
I’m not really convinced by your evidence that there’s a widespread belief that EA is worse than other communities. Your first quote clearly says this, I grant. The other quotes seem to me to all be saying something more like “this community has this problem”, and “I wish this community was better / had hoped that it would be better”, without saying “we are worse than relevant comparisons”, much less “we are worse than relevant comparisons because of intrinsic aspects of our culture”.

FWIW, my belief, and I’m sure I’m not the only one, is more like: EA handles this badly, but also handling this badly is widespread. I don’t think I’ve ever been in a community that I was confident had a good answer for this. EA should address this within its own community, because that’s where we understand what’s happening best and have the best tools for addressing it. This is true regardless of whether EA is worse, the same, or better than its peers: it’s true for as long as it seems feasible to do better than we’re doing.

In the end, interventions should be decided on their merits, and I think we should tip the balance away from the abstract and towards the concrete questions of what specific things we should or shouldn’t do.

Ben Millwood🔸 4 May 2025 13:57 UTC
2 points
0 ∶ 0
in reply to: Yarrow Bouchard 🔸’s comment on: What novel, actionable advice does longtermism offer?

An incredibly mainstream view is to care about everyone alive today and everyone who will be born in the next 100 years. I have to imagine over 90% of people in the world would agree to that view or a view very close to that if you asked them.

I think we have an empirical disagreement here. If I felt strongly motivated to try to persuade you about this, I would go try to find studies about it; I suspect we may not even have 90% agreement on “everyone alive today is worthy of moral concern”, and I would strongly guess we don’t have that level of agreement on caring about people who will be born 50 years from now. (Although I would also guess that many people just don’t think about this kind of question very much and aren’t guaranteed to have very clear or consistent answers.)

Even if people agreed with the premises, we could try to justify longtermism as arguing that the consequences of this belief are underexplored, though I hear you that you don’t see a lot of neglected consequences.

At this point, though, I’m not actually that invested in trying to champion longtermism specifically, so I’m not the right person to defend it to you here. Let’s fix x-risk and check in about it after that :)

Ben Millwood🔸 1 Apr 2025 18:24 UTC
3 points
0 ∶ 0
on: Against Doing Things
forgive the self-promotion but here’s a related Facebook post I made:

The law of conservation of expected evidence, E(E(X|Y)) = E(X), essentially states that you can’t “expect to change your mind”, in the sense that, if you already thought that your estimate of (say) some intervention’s cost-effectiveness would go up by an average of Z after reading this study, then your EV should already have been Z higher before you read it. You should be balanced (in EV terms) between the possible outcomes that would be positive surprises and negative surprises, otherwise you’re just not calculating your EVs correctly.

Anyway, let’s take X to be global future welfare, and Y to be the consequences of some action you take. E(E(X|Y)) = E(X) means that the average global well-being given the outcome of your action is exactly the same as the average global well-being without the outcome of your action. So why did you bother doing it?

Ben Millwood🔸 1 Apr 2025 18:10 UTC
6 points
0 ∶ 0
on: Important Update: Pivotal(TM) Trademark Announcement
I trust that you’ll enforce this trademark against anyone who takes any actions with an unduly large impact on the world, requiring them to first apply for a license to do so.

Ben Millwood🔸 1 Apr 2025 17:57 UTC
40 points
3 ∶ 0
on: What if I’m not open to feedback?
This got me thinking:
no name name
feedback anonymous form normal
no feedback shut up ???
Have you considered making a form where people can submit their names and nothing else?
What links here?
- If No One Builds It, Everyone Dies by Nematode Lover (1 Apr 2026 8:55 UTC; 20 points)

Ben Millwood🔸 22 Feb 2025 17:24 UTC
5 points
2 ∶ 0
in reply to: Arthur Malone🔸’s comment on: Joseph Lemien’s Shortform
Not that it’s super important, but TVTropes didn’t invent the phrase (nor do they claim they did), it’s from Warhammer 40,000.

Ben Millwood🔸 22 Feb 2025 17:19 UTC
10 points
6 ∶ 2
on: Sinergia Review Follow-Up: Everyone Makes Mistakes, Our Hope Is That Mistakes Are Addressed
I downvoted this because I think this isn’t independently valuable / separate enough from your existing posts to merit a new, separate post. I think it would have been better as a comment on your existing posts (and as I’ve said on a post by someone else about your reviews, I think we’re better off consolidating the discussion in one place).

That said, I think the sentiments expressed here are pretty reasonable, and I would have upvoted this in comment form I think.

Ben Millwood🔸 22 Feb 2025 17:06 UTC
9 points
2 ∶ 0
on: Sinergia Animal review by Vetted Causes / Animal Charity Evaluators—looking for opinions
They posted about their review of Sinergia on the forum already: https://forum.effectivealtruism.org/posts/YYrC2ZR5pnrYCdSLt/sinergia-ace-top-charity-makes-false-claims-about-helping

I suggest we concentrate discussion there and not here.

Ben Millwood🔸 22 Feb 2025 9:50 UTC
4 points
1 ∶ 0
on: SWE vs AIS

Someone on the forum said there were ballpark 70 AI safety roles in 2023

Just to note that the UK AI Security Institute employs more than 50 technical staff by itself and I forget how many non-technical staff, so this number may be due an update.

Ben Millwood🔸 11 Feb 2025 9:08 UTC
15 points
3 ∶ 0
on: The standard case for delaying AI appears to rest on non-utilitarian assumptions
This doesn’t seem right to me because I think it’s popular among those concerned with the longer term future to expect it to be populated with emulated humans, which clearly isn’t a continuation of the genetic legacy of humans, so I feel pretty confident that it’s something else about humanity that people want to preserve against AI. (I’m not here to defend this particular vision of the future beyond noting that people like Holden Karnofsky have written about it, so it’s not exactly niche.)

You say that expecting AI to have worse goals than humans would require studying things like what the empirical observed goals of AI systems turn out to be, and similar – sure, so in the absence of having done those studies, we should delay our replacement until they can be done. And doing these studies is undermined by the fact that right now the state of our knowledge on how to reliably determine what an AI is thinking is pretty bad, and it will only get worse as they develop their abilities to strategise and lie. Solving these problems would be a major piece of what people are looking for in alignment research, and precisely the kind of thing it seems worth delaying AI progress for.

Ben Millwood🔸 1 Feb 2025 18:47 UTC
3 points
0 ∶ 0
on: Consider keeping your threat models private.
another opportunity for me to shill my LessWrong writing posing this question: Should we exclude alignment research from LLM training datasets?

I don’t have a lot of time to spend on this, but this post has inspired me to take a little time to figure out whether I can propose or implement some controls (likely: making posts visible to logged-in users only) in ForumMagnum (the software underlying the EA Forum, LW, and the Alignment Forum)

edit: https://github.com/ForumMagnum/ForumMagnum/issues/10345

Ben Millwood🔸 18 Jan 2025 20:41 UTC
9 points
1 ∶ 0
on: Should we try harder to appeal to conservatives?
See also:
(Perhaps there should be a forum tag for this issue specifically, idk)

Ben Millwood🔸 2 Jan 2025 20:00 UTC
12 points
3 ∶ 1
in reply to: Marcus Abramovitch 🔸’s comment on: Funding Diversification for Mid-Large EA Organizations is Nearly Impossible in the Short-Medium Term
I agree overall but I want to add that becoming dependent on non-EA donors could put you under pressure to do more non-EA things / less EA things—either party could pull the other towards themselves.

Ben Millwood🔸 22 Dec 2024 3:05 UTC
4 points
1 ∶ 0
in reply to: Milli🔸’s comment on: The Most Valuable Dollars Aren’t Owned By Us
Keep in mind that you’re not coercing them to switch their donations, just persuading them. That means you can use the fact that they were persuaded as evidence that you were on the right side of the argument. You being too convinced of your own opinion isn’t a problem unless other people are also somehow too convinced of it, and I don’t see why they would be.

Ben Millwood🔸 22 Dec 2024 2:54 UTC
3 points
1 ∶ 0
in reply to: Milli🔸’s comment on: The Most Valuable Dollars Aren’t Owned By Us
I think that EA donors are likely to be unusual in this respect—you’re pre-selecting for people who have signed up for a culture of doing what’s best even when it wasn’t what they thought it was before.

I guess also I think that my arguments for animal welfare charities are at their heart EA-style arguments, so I’m getting a big boost to my likelihood of persuading someone by knowing that they’re the kind of person who appreciates EA-style arguments.

	no name	name
feedback	anonymous form	normal
no feedback	shut up	???