Owen Cotton-Barratt

Hu­man Dig­nity: a review

• Owen Cotton-Barratt 26 Oct 2025 15:00 UTC

  2 points

  0 ∶ 0

  in reply to: OscarD🔸’s comment on: How good would a CCP-dom­i­nated AI fu­ture be?

  Yeah roughly the thought is “assuming concentrated power, it matters what the key powerful actors will do” (the liberal democracy comment was an aside saying that I think we should be conditioning on concentrated power).

  And then for making educated guesses about what the key powerful actors will do, it seems especially important to me what their attitudes will be at a meta-level: how they prefer to work out what to do, etc.

• Owen Cotton-Barratt 23 Oct 2025 0:53 UTC

  4 points

  1 ∶ 0

  on: How good would a CCP-dom­i­nated AI fu­ture be?

  I might have thought that some of the most important factors would be things like:

  • How likely is leadership to pursue intelligence enhancement, given technological opportunity?

  • How likely is leadership to pursue wisdom enhancement, given technological opportunity?

  (Roughly because: either power is broadly distributed, in which case your comments about liberal democracy don’t seem to have so much bite; or it’s not, in which case it’s really the values of leadership that matter.) But I’m not sure you really touch on these. Interested if you have thoughts.

• Owen Cotton-Barratt 21 Oct 2025 14:25 UTC

  2 points

  0 ∶ 0

  in reply to: Emergence101’s comment on: Per­sis­tence, Not Pro­jec­tion: The Case for Loop Main­te­nance over Longtermism

  Thanks AJ!

  My impression is that although your essay frames this as a deep disagreement, in fact you’re reacting to something that we’re not saying. I basically agree with the heart of the content here—that there are serious failure modes to be scared of if attempting to orient to the long term, and that something like loop-preservation is (along with the various more prosaic welfare goods we discussed) essential for the health of even a strict longtermist society.

  However, I think that what we wrote may have been compatible with the view that you have such a negative reaction to, and at minimum I wish that we’d spent some more words exploring this kind of dynamic. So I appreciate your response.

• Owen Cotton-Barratt 1 Sep 2025 16:09 UTC

  6 points

  1 ∶ 0

  in reply to: Will Aldred’s comment on: The ‘com­mu­nity’ tag is problematic

  That makes sense!

  (I’m curious how much you’ve invested in giving them detailed prompts about what information to assess in applying particular tags, or even more structured workflows, vs just taking smart models and seeing if they can one-shot it; but I don’t really need to know any of this.)

• Owen Cotton-Barratt 30 Aug 2025 23:56 UTC

  8 points

  2 ∶ 4

  in reply to: Jason’s comment on: The ‘com­mu­nity’ tag is problematic

  If you want independent criteria-based judgements, it might realistically be a good option to have the judgements made by an LLM—with the benefit of having the classification instantly (as a bonus you could publish the prompt used, so the judgements would be easier for people to audit).

• Owen Cotton-Barratt 10 Aug 2025 23:32 UTC

  6 points

  1 ∶ 0

  in reply to: David T’s comment on: EA as An­tichrist: Un­der­stand­ing Peter Thiel

  Ok thanks I think it’s fair to call me on this (I realise the question of what Thiel actually thinks is not super interesting to me, compared to “does this critique contain inspiration for things to be aware of that I wasn’t previously really tracking”; but get that most people probably aren’t orienting similarly, and I was kind of assuming that they were when I suggested this was why it was getting sympathy).

  I do think though that there’s a more nuanced point here than “trying too hard to do good can result in harm”. It’s more like “over-claiming about how to do good can result in harm”. For a caricature to make the point cleanly: suppose EA really just promoted bednets, and basically told everyone that what it meant to be good was to give more money to bednets. I think it’s easy to see how this gaining a lot of memetic influence (bednet cults; big bednet, etc.) could end up being destructive (even if bednets are great).

  I think that EA is at least conceivably vulnerable to more subtle versions of the same mistake. And that that is worth being vigilant against. (Note this is only really a mistake that comes up for ideas that are so self-recommending that they lead to something like strategic movement-building around the ideas.)

• Owen Cotton-Barratt 10 Aug 2025 16:45 UTC

  6 points

  0 ∶ 0

  in reply to: Jason’s comment on: EA as An­tichrist: Un­der­stand­ing Peter Thiel

  I think that the theology is largely a distraction from the reason this is attracting sympathy, which I’d guess to be more like:

  • If you have some ideas which are pretty good, or even very good, but they present as though they’re the answer needed for everything, and they’re not, that could be quite destructive (and potentially very-net-bad, even while the ideas were originally obviously-good)

  • This is at least a plausible failure mode for EA, and correspondingly worth some attention/​wariness

  • This kind of concern hasn’t gotten much airtime before (and is perhaps easier to express and understand as a serious possibility with some of the language-that-I-interpret-metaphorically);

Embed­ded Altru­ism [slides]

• Owen Cotton-Barratt 26 Jun 2025 17:08 UTC

  2 points

  0 ∶ 0

  in reply to: Manuel Del Río Rodríguez 🔹’s comment on: De­bate: Mo­ral­ity is Objective

  is that you feel that moral statements are not as evidently subjective as say, ‘Vanilla ice-cream is the best flavor’ but not as objective as, say ‘An electron has a negative charge’, as living in some space of in-betweeness with respect to those two extremes

  I think that’s roughly right. I think that they are unlikely to be more objective than “blue is a more natural concept than grue”, but that there’s a good chance that they’re about the same as that (and my gut take is that that’s pretty far towards the electron end of the spectrum; but perhaps I’m confused).

  I’d say again, an electron doesn’t care for what a human or any other creature thinks about its electric charge.

  Yeah, but I think that e.g. facts about economics are in some sense contingent on the thinking of people, but are not contingent on what particular people think, and I think that something similar could be true of morality.

  I, on the contrary, don’t feel like there could be ‘moral experts’

  The cleanest example I might give is that if I had a message from my near-future self saying “hey I’ve thought really hard about this issue and I really think X is right, sorry I don’t have time to unpack all of that”, I’d be pretty inclined to defer. I wonder if you feel differently?

  I don’t think that moral philosophers in our society are necessarily hitting the bar I would like for “moral expert”. I also don’t think that people who are genuinely experts in morality would necessarily act according to moral values. (I’m not sure that these points are very important.)

• Owen Cotton-Barratt 26 Jun 2025 13:47 UTC

  4 points

  1 ∶ 0

  in reply to: Robi Rahman🔸’s comment on: De­bate: Mo­ral­ity is Objective

  See my response to Manuel—I don’t think this is “proving moral realism”, but I do think it would be pointing at something deeper and closer-to-objective than “happen to have the same opinions”.

• Owen Cotton-Barratt 26 Jun 2025 13:45 UTC

  4 points

  0 ∶ 0

  in reply to: Manuel Del Río Rodríguez 🔹’s comment on: De­bate: Mo­ral­ity is Objective

  I’m not sure what exactly “true” means here.

  Here are some senses in which it would make morality feel “more objective” rather than “more subjective”:

  • I can have the experience of having a view, and then hearing an argument, and updating. My stance towards my previous view then feels more like “oh, I was mistaken” (like if I’d made a mathematical error) rather than “oh, my view changed” (like getting myself to like the taste of avocado when I didn’t used to).

  • There can exist “moral experts”, whom we would want to consult on matters of morality. Broadly, we should expect our future views to update towards those of smart careful thinkers who’ve engaged with the questions a lot.

  • It’s possible that the norms various civilizations converge on represent something like “the optimal(/​efficient?/​robust?) way for society to self-organize”

    • I don’t think this is exactly “independent of human or alien minds”, but it also very much doesn’t feel “purely subjective”

  I don’t really believe there’s anything more deeply metaphysical than that going on with morality^[1], but I do think that there’s a lot that’s important in the above bullets, and that moral realist positions often feel vibewise “more correct” than antirealist positions (in terms of what they imply for real-world actions), even though the antirealist positions feel technically “more correct”.

  1. ^

     I guess: there’s also some possibility of getting more convergence for acausal reasons rather than just evolution towards efficiency. I do think this is real, but it mostly feels like a distraction here so I’ll ignore it.

  What links here?

  • Owen Cotton-Barratt's comment on De­bate: Mo­ral­ity is Objective by Bentham's Bulldog (26 Jun 2025 13:47 UTC; 4 points)

• Owen Cotton-Barratt 26 Jun 2025 9:46 UTC

  4 points

  0 ∶ 2

  in reply to: Neel Nanda’s comment on: De­bate: Mo­ral­ity is Objective

  Locally, I think that often there will be some cluster of less controversial common values like “caring about the flourishing of society” which can be used to derive something like locally-objective conclusions about moral questions (like whether X is wrong).

  Globally, an operationalization of morality being objective might be something like “among civilizations of evolved beings in the multiverse, there’s a decently big attractor state of moral norms that a lot of the civilizations eventually converge on”.

• Owen Cotton-Barratt 17 Jun 2025 0:09 UTC

  5 points

  1 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  Ok but jtbc that characterization of “affronted” is not the hypothesis I was offering (I don’t want to say it wasn’t a part of the downvoting, but I’d guess a minority).

  I would personally kind of like it if people actively explored angles on things more. But man, there are so many things to read on AI these days that I do kind of understand when people haven’t spent time considering things I regard as critical path (maybe I should complain more!), and I honestly find it’s hard to too much fault people for using “did it seem wrong near the start in a way that makes it harder to think” as a heuristic for how deeply to engage with material.

• Owen Cotton-Barratt 16 Jun 2025 9:53 UTC

  2 points

  0 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  I’m curious whether you’re closer to angry that someone might read your opening paragraph as saying “you should discard the concept of warning shots” or angry that they might disagree-vote if they read it that way (or something else).

• Owen Cotton-Barratt 15 Jun 2025 23:13 UTC

  19 points

  2 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  Not sure quite what to say here. I think your post was valuable and that’s why I upvoted it. You were expressing confusion about why anyone would disagree, and I was venturing a guess.

  I don’t think gentleness = ego service (it’s an absence of violence, not a positive thing). But also I don’t think you owe people gentleness. However, I do think that when you’re not gentle (especially ontologically gentle) you make it harder for people to hear you. Not because of emotional responses getting in the way (though I’m sure that happens sometimes), but literally because there’s more cognitive work for them to do in translating to their perspective. Sometimes you should bite that bullet! But by the same token that you don’t owe people gentleness, they don’t owe you the work to understand what you’re saying.

• Owen Cotton-Barratt 13 Jun 2025 20:45 UTC

  6 points

  4 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  By “ontologically ungentle” I mean (roughly) it feels like you’re trying to reach into my mind and tell me that my words/​concepts are wrong. As opposed to writing which just tells me that my beliefs are wrong (which might still be epistemically ungentle), or language which just provides evidence without making claims that could be controversial (gentle in this sense, kind of NVC-style).

  I do feel a bit of this ungentleness in that opening paragraph towards my own ontology, and I think it put me more on edge reading the rest of the post. But as I said, I didn’t disagree-vote; I was just trying to guess why others might have.

• Owen Cotton-Barratt 13 Jun 2025 9:48 UTC

  5 points

  2 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  Right … so actually I think you’re just doing pretty well at this in the latter part of the article.

  But at the start you say things like:

  There’s this fantasy of easy, free support for the AI Safety position coming from what’s commonly called a “warning shot”. The idea is that AI will cause smaller disasters before it causes a really big one, and that when people see this they will realize we’ve been right all along and easily do what we suggest.

  What this paragraph seems to do is to push the error-in-beliefs that you’re complaining about down into the very concept of “warning shot”. It seems implicitly to be telling people “hey you may have this concept, but it’s destructive, so please get rid of it”. And I don’t think even you agree with that!
  
  This might instead have been written something like:

  People in the AI safety community like to talk about “warning shots”—small disasters that may make it easier for people to wake up to the risks and take appropriate action. There’s a real phenomenon here, and it’s worth thinking about! But the way it’s often talked about is like a fantasy of easy, free support for the AI Safety position—when there’s a small disaster everyone will realize we’ve been right all along and easily do what we suggest.

  Actually I think that that opening paragraph was doing more than the title to make me think the post was ontologically ungentle (although they’re reinforcing—like that paragraph shifts the natural way that I read the title).

• Owen Cotton-Barratt 13 Jun 2025 9:15 UTC

  2 points

  0 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  Honestly, maybe you should try telling me? Like, just write a paragraph or two on what you think is valuable about the concept /​ where you would think it’s appropriate to be applying it?

  (Not trying to be clever! I started trying to think about what I would write here and mostly ended up thinking “hmm I bet this is stuff Holly would think is obvious”, and to the extent that I may believe you’re missing something, it might be easiest to triangulate by hearing your summary of what the key points in favour are.)

• Owen Cotton-Barratt 13 Jun 2025 0:01 UTC

  14 points

  3 ∶ 0

  in reply to: Holly Elmore ⏸️ 🔸’s comment on: The myth of AI “warn­ing shots” as cavalry

  I upvoted and didn’t disagree vote the original post (and generally agree with you on a bunch of the object level here!); however, I do feel some urge-towards-expressing-disagreement, which is something like:

  • Less disagreeing with claims; more disagreeing with frames?

  • Like: I feel the discomfort/​disagreement less when you’re talking about what will happen, and more when you’re talking about how people think about warning shots

  • Your post feels something like … intellectually ungenerous? It’s not trying to look for the strongest version of the warning shots frame, it’s looking for a weak version and critiquing that (but it doesn’t seem very self-aware about that)

  • This just makes me feel like things are a bit fraught, and it’s trying to push my ontology around, or something, and I don’t quite like it

  • The title makes me feels especially uneasy in this regard (TBC I don’t think the weak version you’re critiquing is absent from the discourse; but your post reinforces the frame where that’s the core version of the warning shot concept, and I don’t want to reinforce that frame)

  • At the same time I think the post is making several valuable points! (This makes me sort of wish it felt a little ontologically gentler, which would make it easier to feel straightforwardly good about, and easier to link people to)