MichaelDickens

Karma: 6,114

I do independent research on EA topics. I write about whatever seems important, tractable, and interesting (to me).

I have a website: https://mdickens.me/ Much of the content on my website gets cross-posted to the EA Forum, but I also write about some non-EA stuff like [investing](https://mdickens.me/category/finance/) and [fitness](https://mdickens.me/category/fitness/).

My favorite things that I’ve written: https://mdickens.me/favorite-posts/

I used to work as a software developer at Affirm.

MichaelDickens Jul 9, 2025, 7:06 PM
6 points
3 ∶ 0
in reply to: Alistair Stewart’s comment on: A shallow review of what transformative AI means for animal welfare
Not OP but I would say that if we end up with an ASI that can misunderstand values in that kind of way, then it will almost certainly wipe out humanity anyway.

That is the same category of mistake as “please maximize the profit of this paperclip factory” getting interpreted as “convert all available matter into paperclip machines”.

MichaelDickens Jul 9, 2025, 6:57 PM
2 points
0 ∶ 1
in reply to: hmijail’s comment on: hmijail’s Quick takes
I don’t cite LLMs for objective facts.

In casual situations I think it’s basically okay to cite an LLM if you have a good sense of what sorts of facts LLMs are unlikely to hallucinate, namely, well-established facts that are easy to find online (because they appear a lot in the training data). But for those sorts of facts, you can turn on LLM web search and it will find a reliable source for you and then you can cite that source instead.

I think it’s okay to cite LLMs for things along the lines of “I asked Claude for a list of fun things to do in Toronto and here’s what it came up with”.

MichaelDickens Jul 8, 2025, 1:31 AM
8 points
0 ∶ 0
on: A shallow review of what transformative AI means for animal welfare
I like this article. I have reservations about some AI-for-animal-welfare interventions, and this article explains my reservations well. Particularly I am glad that you highlighted the importance of (1) potentially short timelines and (2) that the post-TAI world will probably be very weird.

MichaelDickens Jul 7, 2025, 7:03 PM
2 points
0 ∶ 0
in reply to: tobycrisford 🔸’s comment on: LLMs might already be conscious
Yes, it could well be that an LLM isn’t conscious on a single pass, but it becomes conscious across multiple passes.

This is analogous to the Chinese room argument, but I don’t take the Chinese room argument as a reductio ad absurdum—unless you’re a substance dualist or a panpsychist, I think you have to believe that a conscious being is made up of parts that are not themselves conscious.

(And even under panpsychism I think you still have to believe that the composed being is conscious in a way that the individual parts aren’t? Not sure.)

MichaelDickens Jul 7, 2025, 5:58 PM
2 points
0 ∶ 0
in reply to: tobycrisford 🔸’s comment on: LLMs might already be conscious

I don’t find the Turing test evidence as convincing as you present it here.

Fair enough, I did not actually read the paper! I have talked to LLMs about consciousness and to me they seem pretty good at talking about it.

I agree that if each token you read is generated by a single forward pass through a network of fixed weights, then it seems hard to imagine how there could be any ‘inner life’ behind the words. There is no introspection. But this is not how the new generation of reasoning models work. They create a ‘chain of thought’ before producing an answer, which looks a lot like introspection if you read it!

The chain of thought is still generated via feed-forward next token prediction, right?

A commenter on my blog suggested that LLMs could still be doing enough internally that they are conscious even while generating only one token at a time, which sounds reasonable to me.

MichaelDickens Jul 7, 2025, 5:14 PM
7 points
2 ∶ 0
in reply to: saulius’s comment on: saulius’s Shortform
If you haven’t already, you should consider posting your project ideas!
1. You can get feedback on which ideas seem most promising, so maybe you end up getting those ones done
2. Other people might pick up the ideas, or it could inspire related ideas

MichaelDickens Jul 5, 2025, 9:39 PM
2 points
0 ∶ 0
in reply to: Forumite’s comment on: LLMs might already be conscious
That paper is long and kind of confusing but from skimming for relevant passages, here is how I understood its arguments against computational functionalism:
- Section 3.4: Human brains deviate from Turing machines in that brain states require energy to be maintained and Turing machines are “immortal”. [And I guess the implication is that this is evidence for substrate dependence? But I don’t see why.]
- Section 3.5: Brains might violate informational closure, which basically means that the computations a brain performs might depend on the substrate on which they are performed. Which is evidence that AIs wouldn’t be conscious. [I found this section confusing but it seems unlikely to me that brains violate informational closure, if I understood it correctly.]
- Section 3.6: AI can only be conscious if computational functionalism is true. [That sounds false to me. It could be that some other version of functionalism is true, or panpsychism is true, or perhaps identity theory is true but that both brains and transistors can produce consciousness, or perhaps even dualism is true and AIs are endowed with dualistic consciousness somehow.]
I didn’t understand these arguments very well but I didn’t find them compelling. I think the China brain argument is much stronger although I don’t find it persuasive either. If you’re talking to a black box that contains either a human or a China-brain, then there is no test you can perform to distinguish the two. If the human can say things to you that convince you it’s conscious, then you should also be convinced that the China-brain is conscious.

LLMs might already be conscious

MichaelDickensJul 5, 2025, 7:31 PM

33 points

6 comments2 min readEA link

(mdickens.me)

MichaelDickens Jul 2, 2025, 9:09 PM
4 points
0 ∶ 0
on: Survey of 2,778 AI authors: six parts in pictures

“What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment of the human species?” – median: 5% – mean: 16.2%

“What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment of the human species?” – median: 10% – mean: 19.4%

Am I missing something, or are these answers nonsensical? On my reading, the 2nd outcome is a strict subset of the 1st outcome, so the probability can’t be higher. But the median given probability is twice as high.

MichaelDickens Jul 2, 2025, 6:50 PM
2 points
1 ∶ 0
in reply to: Bentham's Bulldog’s comment on: Summary of John Halstead’s Book-Length Report on Existential Risks From Climate Change

this shouldn’t affect your update from Halstead

Would be true if I had read Halstead’s 437-page report, but I didn’t, I only read the intro + your summary. So if I don’t put high credence in your summary then I don’t know what Halstead’s findings were.

MichaelDickens Jul 2, 2025, 2:53 PM
3 points
2 ∶ 0
in reply to: Charlie_Guthmann’s comment on: Charles_Guthmann’s Shortform
I don’t want to limit my interactions to people who agree with me on certain details because I might be wrong about those details and if I’m wrong then I want to be convinced.

MichaelDickens Jul 2, 2025, 2:51 PM
2 points
0 ∶ 0
in reply to: Yadav’s comment on: Yadav’s Shortform
I’m surprised the vote was so close to unanimous!

MichaelDickens Jun 30, 2025, 11:07 PM
11 points
5 ∶ 0
in reply to: Aditya Nair’s comment on: Don’t Eat Honey
If some crops are much better for insect welfare than others, then I want to know about that.

I agree that it’s too onerous to avoid harm in everything you do. But I think the correct way to handle it isn’t to ignore the harm; it’s to figure out which actions you can take that have the best ratio of harm reduction to personal hardship, and then do those.

It may be that the answer turns out to be something weird like “don’t worry about dairy, but do avoid pumpkins and melons”.

MichaelDickens Jun 28, 2025, 4:59 AM
4 points
0 ∶ 0
in reply to: MichaelDickens’s comment on: What are some criticisms of PauseAI?
Hello, Michael!2024, I believe you are wrong to be concerned about #1 and #2. In the process of writing this post, where I looked at all the high-quality studies I could find, I’ve become more confident that peaceful protests are effective. And PauseAI’s protest tactics are consistent with the tactics seen in the high-quality studies.

MichaelDickens Jun 28, 2025, 4:02 AM
4 points
0 ∶ 0
on: Nuclear war tail risk has been exaggerated?
I obtained a probability of human extinction given insufficient calorie production of 10^-6 (= 1/10^6), considering 1 M years is the typical lifespan of a mammal species

This is a central input to your low estimate but it seems like a non-sequitur to me? 1 million years is the unconditional lifespan. The number you want is the lifespan conditional on insufficient calorie production, which is potentially a very different number.

Put another way, you could also reason that there is zero risk from nuclear war:
1. Humans are a mammal species, therefore the baseline probability of extinction is 1e-6.
2. Based on the reasoning in OP, the probability of extinction conditional on nuclear war is 1e-6.
3. These numbers are exactly the same.
4. Therefore, there is zero chance of extinction from nuclear war.

MichaelDickens Jun 27, 2025, 8:22 PM
28 points
9 ∶ 0
on: SPI: The Litigators That Have Never Litigated a Case
I thought the first review was fine—it didn’t say much, but there wasn’t much to say, and you were asked to write the review so you did what you could.

I think this updated review goes too far in criticizing SPI. To me it reads like you are mad at SPI for asking you not to mention confidentiality and then later mentioning confidentiality itself in a comment, and you wrote this negative review as retribution for making you look bad, not as an objective assessment.

In reality, SPI has never filed a lawsuit, never litigated a case, and legally—never had a “pending case.”

The “in reality” phrasing seems to imply that SPI is lying. I don’t think SPI is lying. I don’t know much about law but my guess is it takes a while to prepare a lawsuit, and that they’re still working on it.

SPI says they “spent $0 in [their] entire first fiscal year (2024-2025),” but in January 2025 SPI’s website already stated that they reform pesticide use, protect endangered insects, and challenge insect factory farming.

This reads like you are objecting to SPI’s use of present tense for activities that are ongoing. The use of present tense seems totally fine to me? When the website says “protect endangered insects”, I read that as “The purpose of our ongoing activities is to protect endangered insects”, whereas you seem to be interpreting it as “We have already protected endangered insects” and you’re indirectly accusing them of lying on that basis. Which I don’t think is reasonable.

It is impossible to reform pesticide use with $0.

This is obviously false. You can do political advocacy on a volunteer basis.

Also, SPI doesn’t have $0. It spent $0 in its first fiscal year, and now has a small but nonzero budget.

Overall this review falls into the same patterns that you’ve been criticized for in previous reviews, where you interpret ambiguous evidence in the least charitable possible light, accusing charities of bad behavior when the accusation isn’t warranted.

MichaelDickens Jun 27, 2025, 4:53 PM
20 points
0 ∶ 0
on: Summary of John Halstead’s Book-Length Report on Existential Risks From Climate Change
Follow-up comment now that I’ve finished reading this post:

I found a few minor inaccuracies from spot-checking specific claims. My spot-checks found a pretty high error rate (about 50%), which unfortunately leads me to believe that that I shouldn’t update my beliefs based on this summary.

I like the genre of “summary of a very long article, written by a third party” and I’m glad you wrote it. My guess is you just rushed this post a bit and weren’t as careful as you should have been.

Forest cover has been increasing for decades.

From Figure 1 in the linked paper, it looks like the rate of deforestation has decreased since ~2000, but it’s still positive.

Climate change will likely cause a few hundred thousand extra people to relocate annually, by raising the frequency of extreme heat.

The linked paper is about sea level rise. From skimming, I don’t think it said anything about displacement due to extreme heat.

Halstead estimates that at the highest end, warming-related conflicts will cause roughly an extra 40,000 deaths by 2100.

Where are you getting this? The closest claim I see is page 389:

This suggests that battle deaths [in Africa] will increase to 40,000 by 2100, other things equal.

Which is a very different claim.

MichaelDickens Jun 27, 2025, 4:34 PM
4 points
0 ∶ 0
on: Summary of John Halstead’s Book-Length Report on Existential Risks From Climate Change
Thank you for writing this, I’ve been putting off reading Halstead’s report for the last three years so this summary was helpful for me!

MichaelDickens Jun 27, 2025, 4:14 PM
4 points
0 ∶ 0
on: Climate Change & Longtermism: new book-length report
Does anyone have the quantitative models from this report? x-risk model is here and fossil fuel consumption model is here but these links both say the sheets have been deleted.

I reached out to John Halstead but he is out of office for the next two weeks so I’m posting here on the off chance that one of the commenters has these models.

MichaelDickens Jun 25, 2025, 7:02 PM
16 points
9 ∶ 3
in reply to: LanceSBush’s comment on: Debate: Morality is Objective
The tone of the beginning of this article—putting “quackery” in the title, the insulting opening line “Bentham’s Newsletter is back at it with bad arguments for moral realism”—makes me think it’s not going to give a fair assessment of the arguments. I didn’t read it for that reason. If you want to persuade people like me, you should skip the insults.