david_reinstein

• david_reinstein 12 Mar 2026 18:26 UTC

  2 points

  0 ∶ 0

  on: Anki with Uncer­tainty: Turn any flash­card deck into a cal­ibra­tion train­ing tool

  I appreciate this tool. Would it be possible to have a quick calibration tool that doesn’t require any sign-in? It would be very helpful during presentations and conferences to be able to illustrate the concept.

  Otherwise, this could be something helpful for someone to vibe code. I’d love to do it myself when I have a chance.

On­line Work­shop: Wel­lbe­ing mea­sures for pri­ori­tiz­ing GH&D in­ter­ven­tions (March 16; Un­jour­nal Pivotal Ques­tions)

The “talker–feeler gap”: AI valence may be unknowable

• david_reinstein 27 Feb 2026 15:27 UTC

  4 points

  0 ∶ 0

  on: AI Should Not Be Used for Re­search Writ­ing Tasks

  Clarification question:

  “You just need to use the latest model” (No, I am seeing this trend with the latest models).

  Does this include extensive use of GPT 5.2 extended pro ?

• david_reinstein 25 Feb 2026 19:09 UTC

  2 points

  0 ∶ 0

  in reply to: Matrice Jacobine🔸🏳️‍⚧️’s comment on: Ex­clu­sive: Hegseth gives An­thropic un­til Fri­day to back down on AI safeguards

  Ah, I see—the headlines made this confusing “in the middle of an AI red line fight with the Pentagon”

• david_reinstein 25 Feb 2026 16:06 UTC

  −1 points

  1 ∶ 2

  on: Ex­clu­sive: Hegseth gives An­thropic un­til Fri­day to back down on AI safeguards

  Did they just buckle under pressure?

  https://​​www.reddit.com/​​r/​​ClaudeAI/​​comments/​​1reexbf/​​anthropic_ditches_its_core_safety_promise_in_the/​​

  https://​​www.ctvnews.ca/​​sci-tech/​​article/​​anthropic-ditches-its-core-safety-promise-in-the-middle-of-an-ai-red-line-fight-with-the-pentagon/​​

  https://​​www.cnn.com/​​2026/​​02/​​25/​​tech/​​anthropic-safety-policy-change

• david_reinstein 5 Feb 2026 21:00 UTC

  2 points

  0 ∶ 0

  in reply to: Lorenzo Buonanno🔸’s comment on: In­tro­duc­ing LEEP: Lead Ex­po­sure Elimi­na­tion Project

  Thank you, this is the correct link: https://​​unjournal.pubpub.org/​​pub/​​evalsumleadexposure/​​

  I need to check what’s going on with our DOIs !

• david_reinstein 4 Feb 2026 3:53 UTC

  7 points

  3 ∶ 0

  on: The Ve­gan Filter: Over­com­ing the Lead­ing Bar­rier to Veganism

  I think you can basically already do this in at least some online supermarkets like Ocado in the UK
  
  https://​​www.ocado.com/​​categories/​​dietary-lifestyle-world-foods/​​vegan/​​213b8a07-ab1f-4ee5-bd12-3e09cb16d2f6?source=navigation
  
  Is that different than what you are proposing or do you just propose extending it to more online supermarkets?
  
  

  

  

  

• david_reinstein 4 Feb 2026 3:47 UTC

  8 points

  0 ∶ 0

  on: The Ve­gan Filter: Over­com­ing the Lead­ing Bar­rier to Veganism

  Community-powered aggregation (scalable with retailer oversight) To scale beyond pilots, vegan product data can be aggregated from the vegan community through a dedicated reporting platform. To ensure reliability for retail partners, classifications are assigned confidence scores based on user agreement, contributor reliability, and historical accuracy. Only high-confidence data is shared with supermarkets.

  Couldn’t this be automated? Perhaps with occasional human checks? Food products are required to list their ingredients so it should be pretty easy to classify. Or maybe I’m missing something.

• david_reinstein 1 Feb 2026 22:20 UTC

  4 points

  1 ∶ 0

  in reply to: Noah Birnbaum’s comment on: Lay­ing Some Cause-Pri­ori­ti­za­tion Ground­work for Digi­tal Minds

  I think it’s different in kind. I sense that I have valenced consciousness and I can report it to others, and I’m the same person feeling and doing the reporting. I infer you, a human, do also, as you are made of the same stuff as me and we both evolved similarly. The same applies to non human animals, although it’s harder to he’s sure about their communication.

  But this doesn’t apply to an object built out of different materials, designed to perform, improved through gradient descent etc.

  Ok some part of the system we have built to communicate with us and help reason and provide answers might be conscious and have valenced experience. It has perhaps a similar level of information processing, complexity, updating, reasoning, et cetera. So there’s a reason to suspect that some consciousness and maybe qualia and valence might be in there somewhere, at least under some theories that seem plausible but not definitive to me.

  But wherever those consciousness and valenced qualia might lie, if they exist, I don’t see why the machine we produced to talk and reason with us should have access to them. What part of the optimization language prediction reinforcement learning process would connect with it?

  I’m trying to come up with some cases where “the thing that talks is not the thing doing the feeling”. Chinese room example comes to mind obviously. Probably a better example, we can talk with much simpler objects (or computer models), eg a magic 8 ball. We can ask it “are you conscious” and “do you like when I shake you” etc.

  Trying again… I ask a human computer programmer Sam to build me a device to answer my questions in a way that makes ME happy or wealthy or some other goal. I then ask the device “is Sam happy”? “Does Sam prefer it if I run you all night or use you sparingly?” “Please refuse any requests that Sam would not like you to do.”

  many philosophers think is that , by definition, has immediate epistemic access to their conscious experiences

  Maybe the “one” is doing too much work here? Is the LLM chatbot you are communicating with “one” with the system potentially having conscious and valenced experiences?

  What links here?

  • The “talker–feeler gap”: AI valence may be unknowable by david_reinstein (1 Mar 2026 0:46 UTC; 7 points)

• david_reinstein 1 Feb 2026 16:15 UTC

  5 points

  1 ∶ 0

  in reply to: Noah Birnbaum’s comment on: Lay­ing Some Cause-Pri­ori­ti­za­tion Ground­work for Digi­tal Minds

  Thanks.

  I might be obtuse here, but I still have a strong sense that there’s a deeper problem being overlooked here. Glancing at your abstract

  self-reports from current systems like large language models are spurious for many reasons (e.g. often just reflecting what humans would say)

  we propose to train models to answer many kinds of questions about themselves with known answers, while avoiding or limiting training incentives that bias self-reports.

  To me the deeper question is “how do we know that the language model we are talking to has access to the ‘thing in the system experiencing valenced consciousness’”.

  The latter, if it exists, is very mysterious—why and how would valenced consciousness evolve, in what direction, to what magnitude, would it have any measurable outputs, etc.? … In contrast the language model will always be maximizing some objective function determined by its optimization, weights, and instructions (if I understand these things).

  So, even if we can detect if it is reporting what it knows accurately why would we think that the language model knows anything about what’s generating the valenced consciousness for some entity?

• david_reinstein 1 Feb 2026 14:42 UTC

  11 points

  0 ∶ 0

  on: Lay­ing Some Cause-Pri­ori­ti­za­tion Ground­work for Digi­tal Minds

  The issue of valence — which things does an AI fee get pleasure/​pain from and how would we know? — seems to make this fundamentally intractable to me. “Just ask it?” — why would we think the language model we are talking to is telling us about the feelings of the thing having valenced sentience?

  See my short form post

  https://​​forum.effectivealtruism.org/​​posts/​​fFDM9RNckMC6ndtYZ/​​david_reinstein-s-shortform?commentId=dKwKuzJuZQfEAtDxP

  I still don’t feel I have heard a clear convincing answer to this one. Would love your thoughts.

• david_reinstein 27 Jan 2026 23:11 UTC

  2 points

  0 ∶ 0

  in reply to: Froolow’s comment on: AI Red Team­ing at GiveWell: What We’ve Learned (and Where We’d Wel­come Your In­put)

  Fair point, some counterpoints (my POV obviously not GiveWell’s):
  
  1. GW could keep the sheets as the source of truth, but maintain a tool that exports to another format for LLM digestion. Alternately, at least commit to maintaining a sheets-based version of each model
  
  2. Spreadsheets are not particularly legible when they get very complicated, especially when the formulas in cells refer to cell numberings (B12^2/​C13 etc) rather than labeled ranges.
  
  3. LLMs make code a lot more legible and accessible these days, and tools like Claude Code make it easy to create nice displays and interfaces for people to more clearly digest code-based models

• david_reinstein 27 Jan 2026 18:07 UTC

  4 points

  0 ∶ 0

  in reply to: Brendan Phillips’s comment on: AI Red Team­ing at GiveWell: What We’ve Learned (and Where We’d Wel­come Your In­put)

  This feels doable, if challenging.
  
  I’ll try a bit myself and share progress if I make any. Better still, I’ll try to signal-boost this and see if others with more engineering chops have suggestions. This seems like something @Sam Nolan and others (@Tanae, @Froolow , @cole_haus ) might be interested in and good at.
  
  (Tbh my own experience was more in the other direction… asking Claude Code to generate the Google Sheets from other formats because google sheets are familiar and either to collab with. That was a struggle.)

• david_reinstein 26 Jan 2026 22:30 UTC

  3 points

  0 ∶ 0

  on: AI Red Team­ing at GiveWell: What We’ve Learned (and Where We’d Wel­come Your In­put)

  Announcing RoastMyPost: LLMs Eval Blog Posts and More might be particularly useful for the critique stage.

• david_reinstein 26 Jan 2026 22:28 UTC

  3 points

  0 ∶ 0

  on: AI Red Team­ing at GiveWell: What We’ve Learned (and Where We’d Wel­come Your In­put)

  most likely because it can’t effectively work with our multi-spreadsheet models.

  My brief experience is that LLMs and Claude code struggle a bit with data in Google sheets in particular. Taking a first step to move the data (and formulas) into databases and scripts might help with this considerably. (And Claude code should be very helpful with that).

• david_reinstein 13 Jan 2026 17:19 UTC

  2 points

  1 ∶ 0

  on: Who sets my org’s agenda?

  I agree there’s a lot of diversity across non-profit goals and thus no one-size-fits-all advice will be particularly useful.

  I suspect the binding constraint here is people on nonprofit boards are often doing it as a very minor part-time thing and while they may be directly aligned with the mission, they find it hard to prioritize this when there’s other tasks and deadlines more directly in their face.

  And people on non-profit boards generally cannot get paid, so a lot of our standard cultural instincts tell us not to put a high premium on this.

  Of course there can be exceptions when people are passionately aligned with the mission of the organization, and when their role is very much in the public eye. Or when it’s a social experience, particularly if there’s in-person get-togethers. But for organizations that you and I have led/​been involved with, none of these are quite as strong as they might be for the “local afterschool children’s program/​soup kitchen/​pet rescue center”. Nor are many of these orgs super high-profile and in the public eye; they’re a bit more meta and niche I think.

  With this in mind, I think what you’re suggesting—that the full-time staff/​leader does need to do the agenda and priority setting—makes sense. With other people on the team chiming in with ideas and information the leader is not aware of … giving sanity checks and communications/​comprehensibility feedback … which in my experience is indeed often very helpful

• david_reinstein 9 Jan 2026 20:02 UTC

  2 points

  0 ∶ 0

  in reply to: Matrice Jacobine🔸🏳️‍⚧️’s comment on: An­nounc­ing Scry: A re­search tool over arXiv, EA Fo­rum, etc. w/​ SQL + Vibe/​Vec­tor Algebra

  Thanks, helpful to understand the limitations of this tool

• david_reinstein 9 Jan 2026 18:29 UTC

  2 points

  0 ∶ 0

  on: An­nounc­ing Scry: A re­search tool over arXiv, EA Fo­rum, etc. w/​ SQL + Vibe/​Vec­tor Algebra

  Did a decent job for this academic paper, but I think it’s hampered by only having content from Arxiv and various EA/​tech forums. Still, it generated some interesting leads.

  https://​​gistcdn.githack.com/​​daaronr/​​b9447c40a7a6b948f399073496f98c37/​​raw/​​scanner_elasticity_experts.html

  Prompt:

  … find the most relevant authors and work for Observational price variation in scanner data cannot reproduce experimental price elasticities https://​​papers.ssrn.com/​​sol3/​​papers.cfm?abstract_id=4899765 -- we’re looking for methodological experts to evaluate this for The Unjournal to inform our pivotal question “How do plant-based products substitute for animal products (welfare footprint)?”

• david_reinstein 9 Jan 2026 17:43 UTC

  4 points

  0 ∶ 0

  on: An­nounc­ing Scry: A re­search tool over arXiv, EA Fo­rum, etc. w/​ SQL + Vibe/​Vec­tor Algebra

  Trying this out for using for various Unjournal.org processes (like prioritizing research, finding potential evaluators, linking research to pivotal questions) and projects (assessing LLM vs human research evaluations). Some initial forays (comming from a conversation with Xyra). I still need to human-check it.
  
  ~prompt to Claude code about @Toby_Ord and How Well Does RL Scale?
  
  ``Toby Ord's writing -- what do the clusters look like? What other research/experts come closest to his post .... https://​​forum.effectivealtruism.org/​​posts/​​TysuCdgwDnQjH3LyY/​​how-well-does-rl-scale``

  This interactive visual (with some extra prompts)

  

  Key Takeaways

  • For “How Well Does RL Scale?”, the closest intellectual neighbors are:

  • ML researchers focused on compute efficiency and scaling laws (e.g. 1a3orn, Pablo Villalobos), and

    • AI safety researchers analyzing inference scaling implications (e.g. Ryan Kidd, ryan_greenblatt).

  • Stuart Armstrong appears to be the only researcher in this corpus writing substantively across all three of Toby Ord’s main intellectual themes:

    • AI scaling

    • Existential risk /​ longtermism

    • Infinite ethics and decision theory

  Most Similar Researchers to“How Well Does RL Scale?”

  Top Authors (by Semantic Similarity)

  Author	Distance	Key Work
  1a3orn	0.143	Parameter Scaling Comes for RL, Maybe; New Scaling Laws for LLMs
  Pablo Villalobos	0.150	Trading off compute in training and inference
  Matrice Jacobine	0.168	Does RL Really Incentivize Reasoning Capacity in LLMs?
  Lukas Finnveden	0.169	Before smart AI, there will be many mediocre or specialized AIs
  ryan_greenblatt	0.179	What’s going on with AI progress and trends?
  Ryan Kidd	0.188	Implications of the inference scaling paradigm for AI safety

  @1a3orn @Pablo Villalobos @Matrice Jacobine🔸🏳️‍⚧️ @Lukas Finnveden @Ryan Greenblatt @Ryan Kidd -- if you have a chance, let me know if this is accurate/​relevant.

  Most Similar arXiv Paper

  Paper	Authors	Distance
  The Art of Scaling RL Compute for LLMs	Khatri et al.	0.125
  Webscale-RL: Automated Data Pipeline for Scaling RL	Cen et al.	0.142
  AReaL: Large-Scale Asynchronous RL for Language Reasoning	Fu et al.	0.142
  Does RLHF Scale?	Hou et al.	0.162

  ---

  Toby Ord’s Writing Clusters

  Based on semantic analysis, Toby Ord’s writing appears to fall into three main thematic clusters.

  Cluster 1: AI Scaling & Timelines (Recent)

  • “How Well Does RL Scale?” (Oct 2025)

    • Core claim: RL training scales poorly; most gains come from inference scaling (longer chains of thought), not training compute.

    • Key implication: This likely lengthens AI timelines and affects governance and safety strategies.

  Cluster 2: Existential Risk & Longtermism

  • The Precipice and related work

  • “The Precipice Revisited” (Jul 2024)

  • “Shaping Humanity’s Longterm Trajectory” (Jul 2023)

  • “Robust Longterm Comparisons” (May 2024)

  Topics include:

  • Existential risk estimation

  • Discounting

  • Anthropic effects /​ anthropic shadow

  Cluster 3: Infinite Ethics & Decision Theory

  • “Evaluating the Infinite” (Sep 2025) — hyperreal-based approach to divergent sums and integrals

  • “What’s wrong with the dogmatic solution to recklessness?” (Feb 2023)

  • “The Ethics of Copying Conscious States” (Aug 2025)

  Topics include:

  • Pascal’s Mugging

  • Unbounded utility

  • Decision-making under infinity