tobycrisford 🔸

Karma: 612

tobycrisford 🔸May 22, 2025, 6:42 PM
1 point
0 ∶ 0
in reply to: Anthony DiGiovanni’s comment on: Thinking Insect Suffering Is The Biggest Deal In The World Is Surprisingly Intuitive
Haha, ok, fair enough, I was not expecting that response!
Your solution (and Karnofsky’s) sound very interesting to me. But I’ll need to read both links in more depth to properly wrap my head around it.
A few questions though:
- Karnofsky’s worked example for applying their multi-model technique leads with: “does this action deviate greatly from ‘normality?’” Why is this not just a more formalized version of the appeal to absurdity heuristic?
- Not everyone is a galaxy-brain philosopher who can come up with complex blogposts like those to explain why giving their wallet to a Pascal mugger is wrong, yet everyone gets the correct (presumably) answer to this thought experiment anyway. And I think most are getting there by using some kind of absurdity heuristic? I think that should count in favour of the usefulness of the appeal to absurdity heuristic! Really feels like there’s a good galaxy-brain meme in this. (I get I’m rolling back here on my early suggestion that we could abandon the absurdity heuristic as soon as just one person could come up with a solution to the problem of pascal’s mugger).
- Back to the actual subject of this post: Do you think the approach outlined in your 2 links could be used as an argument against the overwhelming importance of insect suffering, at least for someone who was extremely uncertain about the likelihood of insect sentience or its intensity?

tobycrisford 🔸May 22, 2025, 7:13 AM
1 point
0 ∶ 1
in reply to: Anthony DiGiovanni’s comment on: Thinking Insect Suffering Is The Biggest Deal In The World Is Surprisingly Intuitive
Would you give your wallet to a pascal mugger?
If yes: Guess what? I am a sorcerer from a parallel universe who has the ability to conjure arbitrary numbers of sentient beings into existence at will, and subject them to extreme torture. You tell me how unlikely you think this claim is. I will then threaten 10x the reciprocal of that number, unless you give me £100. I can send you my details and we can arrange the transfer.
If no: How do you explain this other than by an appeal to absurdity? I would love to know the solution to this problem.
Unless or until we have a better solution to this problem than “that’s absurd”, then I think we have to allow appeals to absurdity, especially when used against an argument that bears some resemblance to this pascal mugger example, at least superficially.

tobycrisford 🔸May 21, 2025, 6:27 AM
11 points
3 ∶ 0
on: Do primitive sentient organisms feel extreme pain? disentangling intensity range and resolution
This is a fascinating analysis, thank you for sharing on the forum!
There’s something that bothers me whenever I read this kind of argument though. And that is: you take for granted that it is possible to put units on pain, and that we should all already know what that means. My question to this is:
If I say “Experience A is 13x as intensely painful as Experience B”, then what does this statement mean?
Where does the number 13 come from and how can I interpret it? It is one thing to put an ordering on different experiences in terms of how painful they are, but it is something very different to put a scale on that ordering so that quantitative comparisons like the above make sense.
I really like the discussion of information theory and complexity in this post, considering the question: how many distinct pain states is it possible for an organism to distinguish, given their complexity? But as you acknowledge, even if you can determine this, you are still left with the problem of deciding what the range is between the bottom and top end of those distinguishable states. I agree with you that it’s not clear that a capacity to distinguish more states should necessarily translate into a capacity for experiencing more intense pain.
When you do discuss the intensity question and what that means, you assume that higher intensity pain corresponds to a greater “signal strength” and will require “additional process units”, but this seems like a massive leap to me! What’s the basis for that assumption? Why couldn’t one organism represent extremely intense pain with less processing energy than another organism? And I think we can’t even start evaluating this question properly until we’ve answered the ‘what do these units even mean?’ question that I posed at the start of my comment.
If you define your units to be related to ‘signal strength’ or ‘processing unit count’, then sure, your methodology is sound. But it would then no longer be clear that units defined in this way have anything to do with ethics.
The only way I know to rigorously define what ‘units’ of pain (or more broadly, of utility) mean, is the von Neumann-Morgensten utility theorem. Here, the definition of “Experience A is 13x as painful as Experience B” is that a rational decision maker would be indifferent between a ¹⁄₁₃ chance of receiving Experience A, and a certainty of receiving Experience B.
This is a definition that can be formalized, and most importantly for thinking about ethics, it is the definition that has most decision relevance. But it is also extremely difficult to figure out how we would go about applying this to primitive sentient organisms. The vNMM utility theorem doesn’t even tell you how to calibrate the respective utility functions of two individual humans (it is undetermined up to linear rescalings), let alone how you would go about doing that between a human and a fly.
When comparing humans, we can plausibly assume that similar kinds of pain have similar value for different individuals, and calibrate their respective utility scales that way. When considering insects, I’m not sure I know where to start, even after reading this fantastic post.

tobycrisford 🔸May 20, 2025, 7:28 AM
11 points
3 ∶ 1
in reply to: Henry Howard🔸’s comment on: Thinking Insect Suffering Is The Biggest Deal In The World Is Surprisingly Intuitive
I’m not sure if I agree with you or not, but I don’t know why you were getting so downvoted for this comment (before I strong up-voted, just to balance things out).
I thought the karma system was supposed to be independent of agreement/disagreement? I want to see your side of the discussion explored in the comments. I don’t think people should be downvoting this kind of objection!
Your point 1 seems like a very good question to me, and I would be interested to read the author’s reply.
Your second point also seems like a reasonable response to the piece, and I’m sure represents what a lot of people would feel, especially if not familiar with EA. The author did a good job of anticipating and responding to lots of potential objections, but I don’t think directly addressed this “doesn’t this lead to absurd conclusions?” objection.
The whole argument does feel like it resembles a Pascal’s mugging, in the same vein as strong-longtermism. When you try to do expected value maximization using Bayesian subjective probabilities (e.g. around extinction risk or likelihood of insect sentience or intensity of insect experience), and then start considering situations with huge amounts of potential value, it does seem like a recipe for decision paralysis: “but look how big these numbers are, you can’t be that certain they don’t matter, surely??”

tobycrisford 🔸May 11, 2025, 7:39 AM
1 point
0 ∶ 0
in reply to: tobycrisford 🔸’s comment on: The Case for Insect Consciousness
Would be interested to hear from those who’ve disagreed with this, since I think I’m just pointing out a mathematical mistake? Interested to be corrected if I’ve got something wrong.
Perhaps would help to give some example numbers. Suppose someone assigns, for an insect:
P(react vigorously given pain experienced) = 1
P(react vigorously given no pain experienced) = 0.5
(These numbers seem defensible to me)
This gives you a Bayes factor of 2, when updating your probability that pain is experienced after seeing evidence that insects react vigorously to some negative stimulus. This is not a ‘strong’ update.

tobycrisford 🔸May 4, 2025, 3:47 PM
2 points
0 ∶ 0
in reply to: Yarrow🔸’s comment on: o3
Interesting, thanks!

tobycrisford 🔸May 4, 2025, 8:11 AM
7 points
1 ∶ 0
on: AI can solve all EA problems, so why keep focusing on them?
Even if you’re certain that AGI is only 5 years away and will eradicate all diseases, a lot of children are going to die of malaria in those 5 years. Donating to malaria charities could reduce that number.

tobycrisford 🔸May 1, 2025, 8:08 PM
1 point
0 ∶ 0
in reply to: Yarrow🔸’s comment on: o3
Sure, I think I’ve seen that comment before, and I’m aware Chollet also included loads of caveats in his initial write up of the o3 results.
But going from zero fluid intelligence to non-zero fluid intelligence seems like it should be considered a very significant milestone! Even if the amount of fluid intelligence is still small.
Previously there was a question around whether the new wave of AI models were capable of any fluid intelligence at all. Now, even someone like Chollet has concluded they are, so it just becomes a question of how easily those capabilities can scale?
That’s the way I’m currently thinking about it anyway. Very open to the possibility that the nearness of AGI is still being overhyped.

tobycrisford 🔸Apr 29, 2025, 6:18 AM
2 points
0 ∶ 2
in reply to: Vasco Grilo🔸’s comment on: The Case for Insect Consciousness
I think this is a misapplication of Bayes rule.
What matters is not that the 1st scenario is much more lilely than the 2nd under the hypothesis that pain is experienced (it clearly is). The relevant question is whether the 1st scenario is much more likely under the hypothesis that pain is experienced than under the hypothesis that pain is not experienced (it’s relation to the second scenario is irrelevant, a red herring). And whether this is actually the case is much less clear.
This is what your footnote equation says too, so I’m not disagreeing with that, but I think the way you presented the argument in the text hides this, and might lead someone to misunderstand what it is they are being asked to judge is ‘much more likely’.
You can make an evolutionary argument for why we would expect an animal to react ‘vigorously’ to sustaining damage, and it is not clear why this evolutionary explanation requires the pain to be ‘experienced’. So someone could make an argument that the likelihood of scenario 1 is high under both hypotheses, in which case it should only cause a small change in your priors.
I thought the post was really interesting, thank you for sharing it! It has updated me towards thinking that there’s a higher chance insects might be sentient. But I think things are still a lot more complicated than suggested by this reply.

tobycrisford 🔸Apr 25, 2025, 6:19 AM
3 points
0 ∶ 0
in reply to: Jared Winslow’s comment on: A Primer in Causal Inference: Animal Product Consumption as a Case Study
Thank you for the detailed reply Jared!
It makes sense that including outcome_2 would risk controlling away much of any effect of veganuary on outcome. And your answers to those pre-empted follow up questions make sense to me as well!
But does that then mean my original concern is still valid..? There is still a possibility that a statistically significant coefficient for veganuary_2 in the model might not be causal, but due to a confounder? Even a confounder that was actually measured, like activism exposure?

tobycrisford 🔸Apr 24, 2025, 6:18 AM
4 points
0 ∶ 0
on: A Primer in Causal Inference: Animal Product Consumption as a Case Study
This is a fantastic , clearly written, post. Thank you for writing up and sharing!
In the 3 models, why is outcome_2 not included as a predictor?
I’m just trying to wrap my head around how the 3-wave separation works, but can’t quite follow how the confounders will be controlled for if the treatment is the only variable included from wave 2.
For example, in the first model:
- Suppose ‘activism’ was a confounder for the effect of ‘veganuary’ on ‘outcome’ (so ‘activism’ caused increased ‘veganuary’ exposure, as well as increased ‘outcome’).
- Suppose we have 2 participants with identical Wave 1 responses.
- Between wave 1 and wave 2, the first participant is exposed to ‘activism’, which increases both their ‘veganuary’ and ‘outcome’ values, and this change persists all the way through to Wave 3.
- The first participant now has higher outcome_3 and veganuary_2 than the second participant, with all other predictors in the model equal, so this will lead to a positive coefficient for veganuary_2, even though the relationship between veganuary and outcome is not causal.
I can see how this problem is avoided if outcome_2 is included as a predictor instead (or maybe as well as..?) outcome_1. So maybe this is just a typo..? If so I would be interested in the explanation for whether you need outcome_1 and outcome_2, or if just outcome_2 is enough. I’m finding that quite confusing to think about!

tobycrisford 🔸Apr 2, 2025, 12:01 PM
4 points
1 ∶ 0
in reply to: titotal’s comment on: Large Language Models Pass the Turing Test
Thanks for sharing the original definition! I didn’t realise Turing had defined the parameters so precisely, and that they weren’t actually that strict! I
I probably need to stop saying that AI hasn’t passed the Turing test yet then. I guess it has! You’re right that this ends up being an argument over semantics, but seems fair to let Alan Turing define what the term ‘Turing Test’ should mean.
But I do think that the stricter form of the Turing test defined in that metaculus forecast is still a really useful metric for deciding when AGI has been achieved, whereas this much weaker Turing test probably isn’t.
(Also, for what it’s worth, the business tasks I have in mind here aren’t really ‘complex’, they are the kind of tasks that an average human could quite easily do well on within a 5-minute window, possibly as part of a Turing-test style setup, but LLMs struggle with)

tobycrisford 🔸Apr 2, 2025, 7:22 AM
9 points
1 ∶ 1
on: Large Language Models Pass the Turing Test
I don’t think we should say AI has passed the Turing test until it has passed the test under conditions similar to this:

But I do really like that these researchers have put the test online for people to try!
https://turingtest.live/
I’ve had one conversation as the interrogator, and I was able to easily pick out the human in 2 questions. My opener was:
“Hi, how many words are there in this sentence?”
The AI said ‘8’, I said ‘are you sure?’, and it re-iterated its incorrect answer after claiming to have recounted.
The human said ‘9’, I said ‘are you sure?‘, and they said ‘yes?’.. indicating confusion and annoyance for being challenged on such an obvious question.
Maybe I was paired with one of the worse LLMs… but unless it’s using hidden chain of thought under the hood (which it doesn’t sound like it is) then I don’t think even GPT 4.5 can accurately perform counting tasks without writing out its full working.
My current job involves trying to get LLMs to automate business tasks, and my impression is that current state of the art models are still a fair way from something which is truly indistinguishable from an average human, even when confronted with relatively simple questions! (Not saying they won’t quickly close the gap though, maybe they will!)

tobycrisford 🔸Mar 19, 2025, 1:21 PM
1 point
0 ∶ 0
in reply to: funnyfranco’s comment on: AGI Morality and Why It Is Unlikely to Emerge as a Feature of Superintelligence
Evolution is chaotic and messy, but so is stochastic gradient descent (the word ‘stochastic’ is in the name!) The optimisation function might be clean, but the process we use to search for optimum models is not.
If AGI emerges from the field of machine learning in the state it’s in today, then it won’t be “designed” to pursue a goal, any more than humans were designed. Instead it will emerge from a random process, through billions of tiny updates, and this process will just have been rigged to favour things which do well on some chosen metric.
This seems extremely similar to how humans were created, through evolution by natural selection. In the case of humans, the metric being optimized for was the ability to spread our genes. In AIs, it might be accuracy at predicting the next word, or human helpfulness scores.
The closest things to AGI we have so far do not act with “strict logical efficiency”, or always behave rationally. In fact, logic puzzles are one of the things they particularly struggle with!

tobycrisford 🔸Mar 19, 2025, 7:30 AM
3 points
1 ∶ 0
on: AGI Morality and Why It Is Unlikely to Emerge as a Feature of Superintelligence
I voted ‘disagree’ on this, not because I’m highly confident you are wrong, but because I think things are a lot less straightforward than this. A couple of counterpoints that I think clash with this thesis:
- Human morality may be a consequence of evolution, but modern ‘moral’ behaviour often involves acting in ways which have no evolutionary advantage. For example, lots of EAs make significant sacrifices to help people on the other side of the world, who are outside their community and will never have a chance to reciprocate, or to help non-human animals who we evolved to eat. I think there’s two ways you can take this: (1) the evolutionary explanation of morality is flawed or incomplete, or (2) evolution has given us some generic ability to feel compassion to others which originally helped us to co-operate more effectively, but is now ‘misfiring’ and leading us to e.g. embrace utilitarianism. I think either explanation is good news for morality in AGIs. Moral behaviour may follow naturally from relatively simple ideas or values that we might expect an AGI to have or adopt (especially if we intentionally try to make this happen).
- You draw a distinction between AGI which is “programmed with a goal and will optimise towards that goal” and humans who evolved to survive, but actually these processes seem very similar. Evolutionary pressures select for creatures who excel at a single goal: reproducing, in a very similar way to how ML training algorithms like gradient descent will select for artificial intelligences that excel at a single goal: minimizing some cost function. But a lot of humans have still ended up adopting goals which don’t seem to align with the primary goal (e.g. donating kidneys to strangers, or using contraception), and there’s every reason to expect AGI to be the same (I think in AI safety they use the term ‘mesa-optimization’ to describe this phenomenon...?) Now I think in AI safety this is usually talked about as a bad thing. Maybe AGI could end up being a mesa-optimizer for some bad goal that their designer never considered. But it seems like a lot of your argument rests on there being this big distinction between AI training, and evolution. If the two things are in fact very similar, then that again seems to be a reason for some optimism. Humans were created through an optimization procedure that optimized for a primary goal, but we now often act in moral ways, even if this conflicts with that goal. Maybe the same could happen for AGIs!
To be clear, I don’t think this is a watertight argument that AGIs will be moral, I think it’s an argument for just being really uncertain. For example, maybe utilitarianism is a kind of natural idea that any intelligent being who feels some form of compassion might arrive at (this seems very plausible to me), but maybe a pure utilitarian superintelligence would actually be a bad outcome! Maybe we don’t want the universe filled with organisms on heroin! Or for everyone else to be sacrificed to an AGI utility monster.
I can see lots of reasons for worry, but I think there’s reasons for optimism too.

tobycrisford 🔸Feb 28, 2025, 1:57 PM
6 points
0 ∶ 0
on: How to campaign about the recent UK aid cuts
I’m feeling inspired by Anneliese Dodds’ decision to resign as a government minister over this issue, which is grabbing the headlines today! Before that I’d been feeling very disappointed about the lack of pushback I was seeing in news coverage.
I haven’t written my letter to my MP yet, but I’ve remembered that I am actually a member of the Labour party. Would a letter to my local Labour MP have even more impact if I also cancelled my Labour membership in protest? Ok, I might not be a government minister, I’m just an ordinary party member who hasn’t attended a party event in years, but still, they get some money from me at the moment!
Or would cancelling the membership mean I have less influence on future issues, and so ultimately be counter-productive? Any thoughts?

tobycrisford 🔸Feb 21, 2025, 6:15 PM
4 points
0 ∶ 0
on: Benchmark Performance is a Poor Measure of Generalisable AI Reasoning Capabilities
In addition, o3 was also trained on the public data of ARC-AI, a dataset comprised of abstract visual reasoning problems in the style of Raven’s progressive matrices [52]. When combined with the large amount of targeted research this benchmark has attracted in recent years, the high scores achieved by o3 should not be considered a reliable metric of general reasoning capabilities.
This take seems to contradict Francois Chollet’s own write-up of the o3 ARC results, where he describes the results as:
a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs. o3 is a system capable of adapting to tasks it has never encountered before
(taken from your reference 52 , emphasis mine)
You could write this off as him wanting to talk-up the significance of his own benchmark, but I’m not sure that would be right. He has been very publicly sceptical of the ability of LLMs to scale to general intelligence, so this is a kind of concession from him. And he had already laid the groundwork in his Dwarkesh Patel interview to explain away high ARC performance as cheating if it tackled the problem in the wrong way, cracking it through memorization via an alternative route (e.g. auto-generating millions of ARC-like problems and training on those). He could easily have dismissed the o3 results on those grounds, but chose not to, which made an impression on me (a non-expert trying to decide how to weigh up the opions of different experts). Presumably he is aware that o3 trained on the public dataset, and doesn’t view that as cheating. The public dataset is small, and the problems are explicitly designed to resist memorization, requiring general intelligence. Being told the solution to earlier problems is not supposed to help you solve later problems.
What’s your take on this? Do you disagree with the write up in [52]? Or do you think I’m mischaracterizing his position (there are plenty of caveats outside the bit I selectively quoted as well—so maybe I am)?
The fact that the human-level ARC performance could only be achieved by extremely high inference-time compute costs seems significant too. Why would we get inference time scaling if chain-of-thought consisted of not much more than post-hoc rationalizations, instead of real reasoning?
For context, I used to be pretty sympathetic to the “LLMs do most of the impressive stuff by memorization and are pretty terrible at novel tasks” position, and still think this is a good model for the non-reasoning LLMs, but my views have changed a lot since the reasoning models, particularly because of the ARC results.

tobycrisford 🔸Feb 20, 2025, 8:09 AM
10 points
1 ∶ 0
on: Metaculus Q4 AI Benchmarking: Bots Are Closing The Gap
This is an interesting analysis!
I agree with MaxRa’s point. When I skim read “Metaculus pro forecasters were better than the bot team, but not with statistical significance” I immediately internalised that the message was “bots are getting almost as good as pros” (a message I probably already got from the post title!) and it was only when I forced myself to slow down and read it more carefully that I realised this is not what this result means (for example you could have done this study only using a single question, and this stated result could have been true, but likely not tell you much either way about their relative performance). I only then noticed that both main results were null results. I’m then not sure if this actually supports the ‘Bots are closing the gap’ claim or not..?
The histogram plot is really useful, and the points of reference are helpful too. I’d be interested to know what the histogram would look like if you compared pro human forecasters to average human forecasters on a similar set of questions? How big an effect do we see there? Or maybe to get more directly at what I’m wondering: how do bots compare to average human forecasters? Are they better with statistical significance, or not? Has this study already been done?

tobycrisford 🔸Feb 18, 2025, 6:30 PM
3 points
0 ∶ 0
in reply to: Alfredo Parra 🔸’s comment on: Indirect realism illustrated (and why it matters so much for consciousness debates)
Thanks for the link, I’ve just given your previous post a read. It is great! Extremely well written! Thanks for sharing!
I have a few thoughts on it I thought I’d just share. Would be interested to read a reply but don’t worry if it would be too time consuming.
- I agree that your laser example is a good response to the “replace one neuron at a time” argument, and that at least in the context of that argument, computational complexity does matter. You can’t replace components of a brain with simulated parts if the simulated parts can’t keep up with the rest. If neurons are not individually replaceable, or at least not individually replaceable with something that can match the speed of a real neuron, (and I accept this seems possible) then I agree that the ‘replace one neuron at a time’ thought experiment fails.
- Computational complexity still seems pretty irrelevant for the other thought experiments: whether we can simulate a whole brain on a computer, and whether we can simulate a brain with a pencil and paper. Sure, it’s going to take a very long time to get results, but why does that matter? It’s a thought experiment anyway.
- I agree with you that the answer to the question “is this system conscious?” should be observer independent. But I didn’t really follow why this belief is incompatible with functionalism?
I like the ‘replace one neuron at a time’ thought-experiment, but accept it has flaws. For me, it’s that we could in principle simulate a brain on a digital computer and have it behave identically, that convinces me of functionalism. I can’t grok how some system could behave identically but its thoughts not ‘exist’.

tobycrisford 🔸Feb 17, 2025, 5:36 PM
3 points
0 ∶ 0
in reply to: Alfredo Parra 🔸’s comment on: Indirect realism illustrated (and why it matters so much for consciousness debates)
Thanks for the reply, this definitely helps!
The brain operating according to the known laws of physics doesn’t imply we can simulate it on a modern computer (assuming you mean a digital computer). A trivial example is certain quantum phenomena. Digital hardware doesn’t cut it.
Could you explain what you mean by this..? I wasn’t aware that there were any quantum phenomena that could not be simulated on a digital computer? Where do the non-computable functions appear in quantum theory? (My background: I have a PhD in theoretical physics, which certainly doesn’t make me an expert on this question, but I’d be very surprised if this was true and I’d never heard about it! And I’d be a bit embarrassed if it was a fact considered ‘trivial’ and I was unaware of it!)
There are quantum processes that can’t be simulated efficiently on a digital computer, but that is a different question.