Owen Cotton-Barratt

Karma: 11,005

Owen Cotton-Barratt 1 Oct 2026 22:27 UTC
4 points
0 ∶ 0
in reply to: SoniaAlbrecht 🔸’s comment on: A statement and an apology
Thanks, I appreciate the sentiment. Although I overwhelmingly have myself to blame, I was hurt by this experience; and so it’s a bit cathartic to hear someone saying “sorry” (even if I’m not sure you really have anything to apologise for).
I do want to flag that “miscommunication” feels too trivialising—while miscommunication was an important factor, I also had some problematic attitudes around these topics:
- I felt attraction (fine), felt bad about feeling attraction (perhaps not ideal but pretty normal and fine), and emotionally cast the people I was attracted to as the legitimate judges of whether I was bad (quite problematic; especially given the power differentials).
- Something I dislike about “miscommunication” is that it might be read as suggesting that anyone who had a bad experience interacting with me is equally to blame for that; while I think they might have oriented in ways that would have avoided getting hurt, I think this is better explained primarily in terms of my errors.
(I guess I still have some of the impulse to try to be transparent now things feel less fraught; although I wish I’d tried less aggressively to be transparent in the past.)

Owen Cotton-Barratt 29 Sep 2026 17:19 UTC
3 points
0 ∶ 0
in reply to: Emmanuel Chaves’s comment on: The Formal ITN Framework is Misleading and not Very Useful
Re. 1, I agree that you’re going to have the standard difficulties, but I think that the framework makes it easier to make somewhat-informed guesses (at least if you’re happy to let us assume that returns are approximately logarithmic).
For instance, suppose you’re wondering about contributing $100k to a field that currently spends $50M/year. What will that $100k buy? It’s kind of hard to project counterfactuals. Well, what about doubling the size of the whole field? Or 1000x-ing the size of the field? Sometimes it can be easier to find a scale where effects feel like now you can bring different comparisons or intuitions to bear, as a way of triangulating on a reasonable number.
Re. 2, one point is that if we take log returns as given, then the tractability term for a given field will be the same, regardless of current investment. That feels like a helpful fact for reasoning about things? e.g. if you’re comparing two research fields, you might insert a vibes-based judgement about their relative difficulty (especially if they’re not wildly different in seeming difficulty), without worrying about if that’s somehow getting swamped by differences due to the different sizes of the fields. Similarly it lets you make arguments like most problems fall within a 100x tractability range, which is sometimes enough to feel like it’s doing something useful.
Now I don’t think that log returns is always the right model (especially in domains where we have a good understanding of the interventions, and we’re mostly concerned with their direct effects). But maybe that’s enough to give you some of the flavour of why I think it can get you something helpful, if you do assume it? (I also think there are lots of ways to go wrong within the ITN framework, and endorse Lizka’s post that I linked in my last comment.)

Owen Cotton-Barratt 29 Sep 2026 8:55 UTC
7 points
2 ∶ 1
on: The Formal ITN Framework is Misleading and not Very Useful
The large majority of what you’ve written here is valid, but I think you’re wrong in the conclusion that the model is never (or almost never) useful.
I think this disagreement is going to come down to two things:
1. In my view, direct cost-effectiveness estimates can be extremely hard to do, for anything where a lot of the impact of your actions is indirect and downstream
  - This includes a lot of the most popular intervention classes in EA, e.g. field-building, advocacy, policy, research …
  - When you can make a good direct cost-effectiveness estimate—great! do that. I think that this is more frequent when you’re comparing interventions withing the same cause area, so you’re looking at similar outputs and trying to figure out the most efficient way to buy those outputs
    But a lot of the time, especially when trying to compare different cause areas, I think that this is super hard to do well, and while it’s easy to make a garbage model with the ITN framework, I think it’s typically even easier to make a garbage model of direct cost-effectiveness
2. You kind of acknowledge an important exception in footnote 10, but then don’t properly consider the implications:
[Owen] thinks for problems where success is made by a bunch of small contributions this is theoretically justified. A good example of such a field is research. And studies have indeed shown that research returns are indeed logarithmic. It just doesn’t apply to *every* field
- For tons of problems, we don’t know what the best approach/solution looks like. This means that one of the core activities is research. And I think that’s true for a broad sense of “research”, that can include trying things and learning-by-doing.
- So I agree that this doesn’t apply to every field, but I think that it applies pretty broadly, and is generically a good starting point when we aren’t in a good position to make a good direct cost-effectiveness estimate (which, as per point 1, I think is most of the time)
What links here?
- The Formal ITN Framework is Misleading and not Very Useful by Emmanuel Chaves (28 Sep 2026 21:46 UTC; 4 points)

Why I’m scared of RL

Owen Cotton-Barratt23 Sep 2026 12:11 UTC

57 points

0 comments10 min readEA link

Owen Cotton-Barratt 12 May 2026 8:28 UTC
2 points
0 ∶ 0
in reply to: Toby Tremlett🔹’s comment on: New EA Forum LLM-use policy
Yeah I think that would be an improvement over the current behaviour. I’d still probably prefer something very short (“LLM usage: zero/minimal/moderate/major”) which can be expanded if people want more texture.

Owen Cotton-Barratt 12 May 2026 0:12 UTC
9 points
8 ∶ 2
in reply to: Austin’s comment on: New EA Forum LLM-use policy
I wonder if the disclosures could be non-text by default—e.g. colour-coded with an optional footnote for details.
The thing I’m not liking as a reader is having words to process on this stuff at the start (for me this isn’t just cases where people aren’t following policy; I’ve felt it some about a case where the words were one of the suggested wordings from the policy). Non-text ways to signal could potentially get best-of-both-worlds in terms of reader attention.

Owen Cotton-Barratt 10 May 2026 8:52 UTC
4 points
0 ∶ 3
in reply to: NickLaing’s comment on: New EA Forum LLM-use policy
Ok so I can kind of tune into what you’re saying here, but I also feel kind of uneasy about it. I guess I’d be curious what you make of the following potential arguments:
1. Ingredients are important because we can’t directly discern what’s in food. But with writing we can see exactly what’s there and judge that directly without needing to judge the process. (This perspective would endorse reviews being posted warning people not to read low-quality stuff.)
2. Requiring disclosure is an inappropriate form of thought policing—people should have the right to use whatever cognitive processes and augmentation methods they like, and take responsibility for the words they then share. If this produces LLM garbage it’s not on them to label that up front, but this should have the natural consequence that people stop listening to them.

Owen Cotton-Barratt 8 May 2026 20:51 UTC
4 points
1 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
Ok, so one place the predictions of these theories might come apart is that my theory suggests a norm against impersonating medics, whereas I think yours doesn’t (although maybe I’m just not seeing it; I don’t think I would have said that avoiding torture of prisoners was part of protecting the mechanisms of ending war, although I do kind of see what you mean). I haven’t looked into it at all, but if that norm has emerged independently multiple times that would be suggestive in favour of the broader theory; whereas if it has just emerged once it looks perhaps more potentially-idiosyncratic, which would be suggestive in favour of the narrower theory.

Owen Cotton-Barratt 8 May 2026 20:18 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
I agree that the model I proposed is imprecise; I think this counts against its usefulness but not its validity.
I’m not suggesting this as a thing to advocate for; merely as a descriptive pattern of what the category of war crimes is doing. I think the things which make ending war harder are an important class of really destructive thing, but it seems clarity-obscuring to me to claim that this is definitionally what war crimes are? Rather than giving your thing a new label and then getting to discuss what fraction of war crimes are in that category, and whether there are things in that category which aren’t war crimes (e.g. if torturing POWs counts under your categorization, then why doesn’t conscription count—after all, it damages the “one side runs out of soldiers” mechanism for ending war).

Owen Cotton-Barratt 8 May 2026 19:28 UTC
4 points
1 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
I like the puzzle. But I wonder if you can make your answer even simpler:
- Actions taken in war have some benefit to the perpetrator, and some costs to the larger system of permitting them
- When the ratio between these things gets too extreme, it’s regarded as a war crime
I think this explains the category that you outline (undermining trust in the kind of institutions that could stop the war is super destructive!), but also explains some other cases, e.g. abuse of prisoners, not impersonating medical staff, etc.

Owen Cotton-Barratt 8 May 2026 15:44 UTC
6 points
0 ∶ 0
in reply to: NickLaing’s comment on: New EA Forum LLM-use policy
Hmm, I’ve used LLMs to varying degrees in writing articles. Usually not to the point of writing significant amounts of text, but a case where I think it clearly helped to improve the output is this story: https://strangecities.substack.com/p/some-days-soon
Functionally, I wrote a complete draft, then got Claude to redraft, then I went through and stitched the best bits of the two drafts together (or wrote new versions where that seemed best). (If you thought the original draft was better I’d be interested to hear that: https://docs.google.com/document/d/1icY2wpcgvKszfzHFButKcOwV8B9xMypTAk48kjnOGz0/edit?usp=drivesdk )
(I notice that I’m more likely to find LLMs helpful in drafting things when writing fiction. I think it’s least likely to help when it’s important to convey my precise epistemic status towards the things I’m saying.)

Owen Cotton-Barratt 8 May 2026 3:20 UTC
19 points
5 ∶ 3
in reply to: Will Aldred’s comment on: New EA Forum LLM-use policy
Requiring disclosures to be at the top of the post (rather than e.g. allowing them to be at the bottom) does feel like it’s sending some implicit “this is kind of bad so people need to be warned about it” message, even if it’s in a “recommended uses” section.
Like I think people might reasonably worry about others pre-judging posts with this disclaimer, and hence (perhaps, sometimes) prefer workflows where they don’t need to include the disclaimer, even if this makes their posts worse.
I don’t think there’s an easy answer here—like, presumably the point of the policy is to allow this kind of pre-judging and let people make differently-informed choices about what they engage with. But I think the post kind of papers over this tension.

Owen Cotton-Barratt 15 Apr 2026 15:24 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Coordination without Truth: The Epistemic Cascade
I think we’re still primarily assessing what was said. And you can make your system flexible enough that it doesn’t just entrench existing actors.

Owen Cotton-Barratt 15 Apr 2026 14:09 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Coordination without Truth: The Epistemic Cascade
I think this pushes towards filtering happening via means other than just assessing the argument—e.g. something like persistent reputations (with some occasional sampling to allow new entrants to get out of the zone of being ignored). cf. discussion of how AI could help us keep tabs on the reliability of different actors.

Owen Cotton-Barratt 15 Apr 2026 12:15 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Coordination without Truth: The Epistemic Cascade
I imagine that there’s a relatively cheap filter to work out which arguments to even engage with, and then most effort can still go into engaging with those arguments.

Owen Cotton-Barratt 15 Apr 2026 6:53 UTC
3 points
0 ∶ 0
on: Coordination without Truth: The Epistemic Cascade
I haven’t read the whole post, but tripped up on this:
When the cost of generating arguments approaches zero, their value as traces of reasoning falls, while their value as proxy signals — indirect markers of reliability more amenable to social confirmation — rises.
This seems backwards to me? They’re no longer a costly signal, but they can still be assessed for legitimacy.

AI for epistemics: the good, the bad and the ugly

Forethought13 Apr 2026 17:17 UTC

30 points

4 comments11 min readEA link

(www.forethought.org)

Owen Cotton-Barratt 10 Apr 2026 7:45 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Defense-favoured coordination design sketches
You can have a smart system make inferences from camera visible information.
But yeah, the main use case we had in mind for the monitoring layer was not about these very tricky-to-observe states, but expanding the space of things you can make agreements about (potentially including some high-stakes cases, as I write about at the end of this story: https://strangecities.substack.com/p/some-days-soon).

Owen Cotton-Barratt 9 Apr 2026 18:19 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Defense-favoured coordination design sketches
This is basically the reason I regard this as the most technically challenging of the things we’re presenting here. You eventually want a system which is not just a passive consumer of data, but can actively explore. You may need to give it access to robots with cameras and internet so that it can verify some of the basics of its setup. It might still fear that the entire thing is being spoofed, but I think it’s vastly harder to generate a plausible world that’s robust to the agent exploring and running consistency probes.

Owen Cotton-Barratt 8 Apr 2026 22:39 UTC
2 points
0 ∶ 0
in reply to: Artem Rudnev’s comment on: Defense-favoured coordination design sketches
I’m kind of unsure which of the sketches you’re talking about with this question. Could you ask it of whichever one you feel it’s cleanest for?