Linch

Karma: 28,086

Linch 15 Apr 2026 5:54 UTC
2 points
0 ∶ 0
in reply to: Toby Tremlett🔹’s comment on: Linch’s Shortform
I’m a bit confused by this. Bjartur’s most sub-culture-friendly story, naively (in the sense that it’s chockfull directly of in-group references) is The Company Man, which also happens to be his most popular story, by a considerable margin.
Most of his other stories are only particularly subculture-specific in that they tend to have as their foundational basis a hypothesis like “AI will be a big deal”, which I expect to be much less subculture-specific in the future to the extent this hypothesis is correct.

Linch 15 Apr 2026 4:55 UTC
14 points
0 ∶ 0
on: Linch’s Shortform
I’m starting an internal red-teaming project at Forethought against Forethought’s AI propensity targeting/model character work. The theory of change for internal red-teaming is that someone (me!) spending dedicated focused time on the negative case can find and elaborate on holes that Forethought’s existing feedback processes will miss.
Possible good outcomes from the red-teaming project (roughly in decreasing order of expected impact):
1. I identify and come up with good reasons to wind down Forethought’s work in this field iff it is correct to do so.
  1. Note that “identify” in this case may involve original research, but it may instead be mostly about sourcing the ideas from other people.
2. I identify ways the current direction of propensity targeting work is bad and suggest significant directional changes.
3. People don’t change high-level direction, but I understand and can make the counterarguments to this work legible enough within Forethought, so ppl are aware of the counterarguments and make less mistakes in their research.
4. I change my own mind significantly on the case for this work and decide to double down on it myself.
5. I identify sufficiently concrete cruxes that we can use to decide whether to prioritize this work in the future (eg stop/wind down this work if XYZ happens)
I think I’d find it very helpful to understand why people are skeptical of AI propensity targeting work either in general or by Forethought in particular.
For me, the most useful critiques will elaborate on why the work is net-negative (ideally significantly) or why it’s almost certain to be useless. Though other critiques are welcome as well.

I’d really appreciate comments from people here! Especially helpful is links to longer write-ups/existing comments. I might also want to chat on a call with people if anybody’s interested.
I’ve already read comments here, here, and various critical writings on Anthropic’s Constitution and “soul doc” (which is related though ofc not the same thing)

Linch 13 Apr 2026 17:22 UTC
5 points
2 ∶ 0
on: Linch’s Shortform
wrote a review of Tomas Bjartur’s fiction, a sci-fi writer with many ideas that’s interesting and relevant to this forum (especially AI risk) https://linch.substack.com/p/tomas-bjartur

Linch 12 Apr 2026 1:14 UTC
3 points
1 ∶ 0
in reply to: Clara Torres Latorre 🔸’s comment on: Clara Torres Latorre ️’s Quick takes
I try to avoid downvoting things that are already in the red, personally. Unless it’s very bad.

Linch 10 Apr 2026 18:59 UTC
3 points
1 ∶ 0
in reply to: Pedro Freire’s comment on: Linch’s Shortform
Oh that’s a really good point, thanks. I also get annoyed when people in comments harp on a bad title without providing a better one, instead of engage with the substance of my arguments.

(I thought it’s fine to complain in this case because they clearly benefited a bunch from the equivocation in their title and clear better alternatives were available, whereas when I have bad titles they tend to be clear own goals in the sense that I both got more flak and also less readership than if I had a better title).

Linch 9 Apr 2026 18:13 UTC
4 points
1 ∶ 1
in reply to: titotal’s comment on: Linch’s Shortform
The aforementioned study reported that generative AI adoption in the U.S. has been faster than personal computer (PC) adoption, with 40% of U.S. adults adopting generative AI within two years of the first mass-market product release compared to 20 % within three years for PCs.But this comparison does not account for differences in the intensity of adoption (the number of hours of use) or the high cost of buying a PC compared to accessing generative AI. 14. Alexander Bick, Adam Blandin, and David J. Deming. 2024. The Rapid Adoption of Generative AI. National Bureau of Economic Research.Depending on how we measure adoption, it is quite possible that the adoption of generative AI has been much slower than PC adoption.
re1: I can’t think of a single metric for “PC” or “computer” analogue where you start with <<1% usage (as is the case with LLM-mediated chatbots) and get to >20% in 3 years, so I don’t think the PC analogy is correct. It’s obviously extremely disanalogous/suspicious here where they set up a foil and only criticize the minor problems that makes the analogue look better for LLM adoption speeds when the much more obvious disanalogy makes LLM adoption speeds look worse.
“Point 3 is not even an argument, just a restatement of what they believe” drawing a highly unusual and unmotivated reference class without defending against the most obvious counterarguments and objections is a bad move! Stating reasons for X is not the same as arguing for X against the strongest version of not-X. They do the first; the objection is that they don’t do the second, and the unargued reference class is doing all the work. This is also what I mean by “vibes” doing much more of the argument than you seem to believe.
“Point 5 is not an argument either: they are not to blame for how you interpret their “vibes”. It’s the title of their post! The equivocation is load-bearing for the paper’s reception. If they had titled it “AI as Slow Transformative Technology” or “AI Will Reshape the Economy Over Decades, Not Months,” it would have gotten a fraction of the citations, etc, etc. “The title and framing do the rhetorical work of ‘AI is not a big deal’; the technical content predicts electricity-scale transformation; when talking to journalists or among useful idiots clarification is not needed; when criticized, the authors retreat to the technical content while keeping the rhetorical benefit of the title.
re 6 “Do you think that AI systems are merely cheating on every single benchmark” no i think models are systematically good at easily measurable short time-horizon tasks relative to humans.
First, benchmarks have construct-validity problems even when honestly measured. A benchmark is a sample of tasks chosen to be tractable, verifiable, and gradeable, often with short time horizons (and not requiring long-term planning) The set of tasks with those properties is systematically biased toward what models are good at (at least relative to humans): tasks with crisp answers, short context, well-specified inputs, non-novel circumstances, and clean evaluation criteria^[1].
Second, even setting construct validity aside, optimization pressure on any specific metric degrades that metric’s correlation with the underlying capability, because labs (entirely ~legitimately!) train on data that resembles the benchmark, design architectures that excel at benchmark-shaped problems, and iterate on whatever moves the benchmark number. This is Goodhart’s Law operating normally. Most ppl in AI would not consider this fraud or cheating.
Note that (as I alluded to earlier) my worldview makes different predictions with frozen AI capabilities than N&K make. N&K believes current (and early 2025-era) AI capabilities will cause dramatic shifts in expert labor, just with decades to diffuse. Whereas my perspective (construct-validity issues means models are dramatically good at a few things now but mostly the benchmarks overpredict true ability) says frozen capability would not lead to >~5x changes than we currently observe because the binding constraint is in the parts benchmarks don’t test.
I probably won’t engage further on this thread.
1. ^
  (I have a lot of sympathy towards models having this shape as someone who’s maybe 0.5 sd above average at taking tests relative to my estimation of my actual capabilities, myself).

Linch 8 Apr 2026 22:27 UTC
7 points
5 ∶ 7
on: Linch’s Shortform
Many people hold up ‘AI As Normal Technology’ as a reasonable “normal-people” case against the doomer position. I actually think it’s wrong on a number of ways and falls flat on its own terms. I think I believe this for reasons mostly orthogonal to being a doomer (except inasomuch as being a doomer makes me more interested in thinking about AI). If anybody here is interested in fighting the good fight, it might be valuable to do a Andy Masley-style annilihation of the AI As Normal Technology position, trying to stick to minimally controversial arguments and just destroying their arguments with reference to obvious empirical and logical arguments. I suspect it won’t be very hard. Eg here’s a few obvious reasons they fail:
1. Their central empirical mechanism is already wrong: their story is that AI diffusion will be slow because this is the path of previous technologies like electricity, but consumer and developer adoption of LLMs has been faster than essentially any technology in history (eg Anthropic at 30B ARR)
2. They completely ignore that AI will obviously do a ton to assist in its own diffusion: Even if I take their arguments that diffusion is what matters and I rule out software-only singularity by fiat, I still don’t think I or anybody else should buy their causal mechanisms. Like the single most obvious way in which AI diffusion might be distinct from previous technological changes is afaict unaccounted for in their arguments, even if I presume a diffusion-first model.
3. The reference class is unargued and load-bearing: The whole thesis rests on AI being like electricity or the internet (decades of diffusion) rather than like smartphones, SaaS, or cloud (years).
4. They have no framework that can engage software-only-singularity-style arguments. Their entire ontology is built around physical-world deployment friction. This practically assumes the conclusion!
5. The position is self-undermining for their vibes if you take it literally. 1) If AI really is like electricity, then taken seriously they’re predicting one of the largest economic transformations in human history. 2) Notably they’re predicting this at current levels of AI capabilities. Ie, if AI progress freezes today they’d predict Anthropic’s revenues to massively increase beyond the current 30B ARR. This is a massively big deal!
6. They confuse benchmark-impact gaps with deployment friction (!), when the simpler explanation is benchmark Goodharting and jagged-frontier effects. They believe that the reason models perform well on benchmarks but hasn’t had much more economic impact yet (though, again, note that this has already caused some of the largest and fastest growing companies in history to arise, including in revenue) is due to diffusion dynamics. But obviously the simpler argument is that benchmarks overstate actual AI capability relative to humans.
  1. I don’t think they actually misunderstand this point. The same people who wrote “AI as Normal Technology” wrote “AI As Snake Oil” earlier, seemingly happy to understand the AI capabilities lag benchmarks” position back when it benefitted their arguments.
Overall I think it’s a deeply unserious form of futurism, only held up by Serious Policy People who want to believe in a pre-determined comfortable conclusion.
Should be fun to take down for any of my friends who are bored undergraduates or graduate students interested in destroying bad arguments. Could be a easy way to get a bunch of views on a moderately important topic.

I’m Suing Anthropic for Unauthorized Use of My Personality

Linch2 Apr 2026 1:43 UTC

44 points

0 comments7 min readEA link

Linch 31 Mar 2026 10:43 UTC
2 points
1 ∶ 0
in reply to: huw’s comment on: How did Leopold do? Evaluating Situational Awareness’s predictions
I mostly agree, though it’s not obvious to me that we ought to have sufficient evidence by now re: espionage.

Linch 29 Mar 2026 22:52 UTC
21 points
9 ∶ 1
on: How did Leopold do? Evaluating Situational Awareness’s predictions
AI revenue is his clearest miss: he predicted $100B run rate by mid-2026, and the best is $60B.

Honestly being off by less than a factor of 2 doesn’t seem that bad, especially since it’s not yet April.

Linch 20 Mar 2026 15:47 UTC
2 points
0 ∶ 0
in reply to: NickLaing’s comment on: Broad Timelines
Just the 25th and 75th percentile?

Linch 20 Mar 2026 15:28 UTC
8 points
2 ∶ 1
in reply to: NickLaing’s comment on: Broad Timelines
“I feel like if you’re going to put something out there in the public sphere as a leader in AI, a bit of timeline conservatism might be prudent.”

I see and respect that position, but you can imagine someone saying the opposite: “I feel like if you’re going to put something out there in the public sphere as a leader in AI, it’s probably prudent to warn people of significant risks that happens much sooner than people expect, even if you think it’s less than 50% likely to happen then.”

Linch 20 Mar 2026 14:32 UTC
4 points
0 ∶ 0
in reply to: NickLaing’s comment on: Broad Timelines
Plausibly you can get away with reporting 3-5 numbers.

For 3 numbers, 25th percentile, median, 75th percentile. This is the approach (“interquartile range”) used for reporting SAT acceptance ranges in the US. So we have at least a prior example of a widely reported figure that people don’t think “normal people”/high-schoolers and their parents would have too much trouble understanding.
For 5 numbers, something like 5th percentile, 25th percentile, median, 75th percentile, 95th percentile.
3-5 numbers obviously harder to communicate than 1 number, and less precise than the full distribution. But hopefully it’s clear and useful enough to be good here.

Linch 20 Mar 2026 14:30 UTC
6 points
2 ∶ 1
in reply to: NickLaing’s comment on: Broad Timelines
I think they were optimizing for a combination of concreteness (so there’s an exact story to point to, where the 2027 story is “things go roughly as they expected” whereas 2028 and 2031 were pricing in different types of individually unexpected delays^[1]), and for memetic value.
1. ^
  Compare: My best estimate is that this project will take me 6 months. However, if you ask me to write it out step-by-step, it’d take me 4 months.The 6 months include buffers for various delays, some expected and some unexpected.
  I think for project time estimations as part of a larger plan, the 6 month reply is more useful. But for someone following along on my thinking process, or a manager/collegue/direct report trying to help me optimize, the 4 month step-by-step report might be easier to follow along and/or more useful to critique or improve.

Linch 19 Mar 2026 18:07 UTC
4 points
0 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
wrote it up in more detail on substack: https://linch.substack.com/p/further-moral-goods

Linch 16 Mar 2026 13:44 UTC
4 points
0 ∶ 0
in reply to: David Mathers🔸’s comment on: Linch’s Shortform
Agreed. It’s possible that we/our descendants won’t see much value for extending past blissful experiences even when other axes of value are theoretically possible, in the same way that aliens without conscious experiences would not see any particular reason to privilege qualia (even if they could be convinced that it’s real).

Linch 14 Mar 2026 20:26 UTC
10 points
4 ∶ 0
on: Linch’s Shortform
I think a common mistake for researchers/analysts outside of academia[1] is that they don’t focus enough on trying to make their research popular. Eg they don’t do enough to actively promote their research, or writing their research in a way that’s easy to become popular. I talked to someone (a fairly senior researcher) about this, and he said he doesn’t care about mass outreach given that only cares about his research being built upon by ~5 people. I asked him if he knows who those 5 people are and could email them; he said no.
I think this is a systematic mistake most of the time. It’s true that your impact often routes through a small number of people. However, only some of the time would you know who the decisionmakers are ahead of time (eg X philanthropic fund should fund Y project, B regulator should loosen regulations in C domain), and have a plan for directly reaching them. For the other cases, you probably need to reach at minimum thousands of vaguely-related/vaguely-interested people before the ~5 most relevant people for your research would come across your research.
Furthermore, popularity has other advantages:
- If many people read your writing, it’s more likely someone else can discover empirical mistakes, logical errors, or (on the upside) unexpected connections. If 100 randos read your article, it’s unlikely any of them can discover a critical mistake. This becomes much more likely at 10,000+ randos.
- writing for a semi-popular audience forces some degree of simplicity and a different type of rigor. If you write for “informed people” or “vaguely related experts” as opposed to people in your subsubfield, you have less shared assumptions, and are forced to use less jargon and be more precise about your claims.
- Recruitment and talent attraction. If your research agenda is good, you want other people to work on it. Popular writing is one of the best ways to get other smart people (with or without directly relevant expertise) to notice a problem exists and decide to dedicate time to it.
- Personal career benefits: funders you haven’t heard of before (and who haven’t heard of you) are more likely to discover you and proactively offer funding. Employers working on related fields, or just who like your reasoning style, are more likely to actively recruit you.
Now of course it’s possible to aim too much for popularity, and Goodhart on that. For example, by focusing on research topics that’s popular rather than important, or on research directions/framings that’s memetically fit rather than correct. Obsessing over metrics can also be bad for having the space to explore newer and more confusing ideas.
Nonetheless, on balance I think most researchers should be aware of what makes their research popular and in part gravitate towards that. Maybe I think they should spend >10% of their time on publicizing their work (not including “proof of work” style paper writing, grant applications, etc), whereas instead many ppl seem to spend <5%.
[1] Academia (and for that matter, for-profit research within a company) has this problem less because usually your peer group and potential collaborators in your sub-subfield are more well-defined and known to you. Also, academics care less about impact. Even so, I think ppl are leaving impact (and possibly career success) on the table by not being more popular. Eg if you work in theoretical econ you should aspire to have your theories be applied by applied economists, if you work on the evolutionary dynamics of bees you should want to be read by people working on ants, if you work on themes in Renaissance art history you should aspire to be read by people studying Renaissance political philosophy, etc.
This means (imo) academics should be more willing to have academic blogs and Twitter threads, and tolerate (or even seek out) media coverage of their work.

Linch 14 Mar 2026 17:02 UTC
48 points
1 ∶ 0
on: Linch’s Shortform
“There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy”
One thing I’ve been floating about for a while, and haven’t really seen anybody else deeply explore[1], is what I call “further moral goods”: further axes of moral value as yet inaccessible to us, that is qualitatively not just quantitatively different from anything we’ve observed to date.
For background, I think normal, secular, humans live in 3 conceptually distinct but overlapping worlds:
1. The physical world: matter, energy, atoms, stars, cells. An detached external observer might think that’s all there is to our universe.
2. The mathematical world. Mathematics, logic, abstract structure, rationality, “natural laws.” Even many otherwise-strict “materialists” can see how the mathematical world is conceptually distinct from the physical one: mathematical truths seem conceptually different and perhaps deeper than mere physical facts. And if you’re a robot/present-day LLM, you might just live in the first two worlds[2]. Some Kantians try to ground morality entirely within this world, in the logic of cooperation and strategic interaction.
3. The world of consciousness. The experiential realm. Qualia, subjective experience, “what it’s like to be me.” Most secular moral philosophers treat this as where the real moral action is. A pure hedonic utilitarian might think conscious experience is the only thing that matters, but even other moral philosophies would consider conscious experience extremely important (usually the most important).
For the purposes of this post, I’m not that interested in the delineating between whether these worlds are truly different or just conceptually interesting ways to talk about things (ie I’m not positing a strong position on mathematical platonism or consciousness dualism)
But what’s interesting to me is how these different worlds ground morality/value, what some philosophers would call “axiology.” When people try to solely ground morality in the first two worlds, and even more so when people try to ground morality in the first world alone[3], deep believers in all three worlds (which I think is most people, and most philosophers) think they’re entirely missing the point! It seems almost self-evident that conscious experience is much more important than the arrangement of mere rocks, or bloodless abstract game theory of feeling-less zombies!
But are these the only 3 worlds? Is it possible to have other morally relevant worlds, and in particular worlds that will self-evidently seem so much more important than subjective experience if only we know about them?
Perhaps.
For example, (most) religious people believe they have an answer, :
1. The supernatural world. The world of spirits, Gods, heavens and hells. Religious traditions often claim that divine or transcendent value is qualitatively, not just quantitatively, superior to natural goods. Saying that “heaven is infinite bliss” is a secular/materialist approximation of something much deeper. (Other handles: the ineffable, the sublime)
Now I think the religious people are wrong about the world as we see it today. But do we have strong reason to think that the three worlds as we know them are the only ones left? I think no.
In particular, we have two distinct reasons to think future intelligences can discover other worlds:
A. AIs, including future AIs, will be a distinct type of mind(s) than human mind(s). Just as most people today believe that humans (and other animals) have qualia that present-day AIs do not have, we should also think it’s plausible that different mental architectures in AI will allow them to have moral goods that we cannot experience or perhaps even conceive.
B. Superintelligences (likely digital intelligences, though in theory could also be our posthuman descendants) will be able to search for further moral goods. At some point in the future (if we don’t all die first), it will become trivial to spend more brainpower than has ever existed in all of human science and philosophy combined to search for other sources of moral value. This can come from engineering unique environmental arrangements of matter, unique structures of minds, or something else entirely.
So one day our descendants may discover worlds five, six, and so on: sources of moral value qualitatively distinct and superior to what we have access to, in the same way that grounding morality purely in game theory or entropy feels foolish to most experiencing humans today.
If true, this is a big deal! [4]
This seems overall quite possible to me. But is it probable?
I don’t have a good sense of high likely this all is. Trying to estimate it feels beyond my forecasting or philosophical competence. But it seems plausible enough, and interesting enough, that I wanted to bring it to people’s attention, in case other people have ideas on how to extend it.

Appendix A:

Existing literature: This concept is widespread but undertheorized. Mill’s qualitative distinction among pleasures can point us in this direction; Bostrom’s “Letter from Utopia” is the most vivid articulation (“What I feel is as far beyond feelings as what I think is beyond thoughts”); Danaher (2021) coined “axiological possibility space”; Ord’s The Precipice argues we have “barely begun the ascent” and our investigations of flourishing may be “like astronomy before telescopes.” According to a search from Claude, Nagel, Jackson, and Chalmers “collectively demonstrate that the space of possible conscious experiences vastly exceeds human experience.” Banks’s concept of Subliming: where “the very ideas, the actual concepts of good, of fairness and of justice just ceased to matter”, is the most philosophically precise depiction I’ve seen in science fiction.
[1] Though I’ve seen shades of it in academic philosophy, EA/longtermist writing, science fiction/fantasy, and discussions of religion
[2] This is disputed.
[3] eg entropy as the guiding factor of morality, a la Beff Jezos.
[4] And if false, but convincing enough to be an attractor state for our descendants, this will sadly also be a very big deal.

Linch 13 Mar 2026 12:13 UTC
11 points
0 ∶ 0
on: Linch’s Shortform
On the off chance anybody is both interested in AI news and missed it, Anthropic sued DoW and other government officials/agencies for the supply chain risk designation in DC and Northern Californian Circuit. The full-text of the Northern Californian complaint here:
The primary complaints:
1. First Amendment retaliation. Anthropic alleges that Pentagon officials illegally retaliated against the company for its position on AI safety. They argue that Trump, Hegseth, and others wanted to punish Anthropic for protected speech, citing public social media and other dialogue as evidence that the punishment is ideological in nature.
2. Misuse of the supply chain risk designation. Anthropic was officially designated a supply chain risk, which requires defense contractors to certify they don’t use Claude in their Pentagon work. Anthropic argues that this is a misuse of the SCR designation which Congress intended to stop foreign actors, and that Anthropic clearly does not pose a supply-chain risk in a reading of the law.
3. Lack of Due Process (Fifth Amendment violation). “The Challenged Actions arbitrarily deprive Anthropic of those interests without any process, much less due process.”
4. Ultra vires. Anthropic alleges that the Presidential Directive requiring every federal agency to immediately cease all use of Anthropic’s technology exceeds the limits of the President’s authority as granted by Congress.
5. Administrative Procedural Act. Similar to the above, Anthropic argues that administration violates the administrative procedural act, and the sanctions are not permitted to the relevant agencies as a duty granted to them by Congress.
IANAL etc. in my personal opinion #2 seems very clearcut as a common-language and precedent reading of these things. #1 also seems strong. Sources I randomly skimmed online thought #3-#5 had a good case too, but I don’t have an independent view.
The DC complaint looks less meaty (and I didn’t read it)

Linch 13 Mar 2026 11:58 UTC
3 points
0 ∶ 0
in reply to: Mahdi’s comment on: anormative’s Quick takes
I think they were laughed at enough after the Wired article (from here and elsewhere) that maintaining the previous line was no longer tenable for them.
I also separately think their current stated position is more accurate than the previous one, but I’m just observing that the incentives are a larger fraction of the story behind what they’re saying than what ppl might otherwise be reading them as.

Linch

I’m Su­ing An­thropic for Unau­tho­rized Use of My Personality

I’m Suing Anthropic for Unauthorized Use of My Personality