The object-level argument, as I understand it, is that worries about human-level AI capabilities of the sort that could pose an existential threat are based on a misunderstanding of what is going on under the hood in neural networks. This is what Bender means when she talks about âAI Hypeâ. See for example her paper with Koller âClimbing towards NLUâ for criticisms of attributing some kinds of mental states to neural networks.
The paper you mentioned doesnât seem to discuss existential risk or AGI at all, so I donât see how it could represent the sort of object-level argument against existential risk that Peter is asking for.
Suppose I am very worried that ghosts will steal things out of my closet. It seems like a perfectly object-level argument against my position to provide reasons for thinking that beliefs in paranormal activity are not scientifically respectable. This can be true even if the reasons provided do not mention ghosts.
People like Bender take themselves to be offering reasons for thinking that worries about AGI are not scientifically respectable. This can be true even if the reasons they provide do not mention AGI.
Note that I think Benderâs arguments are bad. But I donât see what is so mysterious about them.
It seems to me that, while the form/âmeaning distinction in this paper is certainly a fascinating one if your interests tend towards philosophy of language, this has very little to say about supposed inherent limitations of language models, and does not affect forecasts of existential risk.
Ok, let me spell it out explicitly. In a section called âLarge LMs: Hype and analysis,â the linked paper says that claims that LLM can âunderstand,â âcomprehend,â and âknowâ are âgross overclaims.â The paper supports this contention by pointing to evidence that âin fact, far from doing the âreasoningâ ostensibly required to complete the tasks, [LLMs] were instead simply more effective at leveraging artifacts in the data than previous approaches.â
Here is where the imagination comes in. Imagine that you think that all mental state attributions to artificial systems are confused in exactly this way. Imagine that you think that artificial neural nets canât reason at all. Now imagine that someone tells you that we should all be very concerned that misaligned superintelligent AI systems will destroy us.
Your response to that would be something like: it is deeply confused to think that superintelligent AI systems are something we need to worry about, and the people who are worried about them simply do not understand what is going on under the hood in machine learning models. Worries about existential risk from superintelligent AI stem from the same kind of confusion as attributing understanding to existing systems: the tendency of people who are not technically literate to anthropomorphize the systems they interact with.
Imagine that you think that artificial neural nets canât reason at all.
Is this a real position that real living intelligent people actually hold, or is it just one of the funny contrarian philosopher beliefs that some philosophers like to around with for fun?
I never denied that they have published their arguments in many places. I just canât find any such arguments that are object-level.
The object-level argument, as I understand it, is that worries about human-level AI capabilities of the sort that could pose an existential threat are based on a misunderstanding of what is going on under the hood in neural networks. This is what Bender means when she talks about âAI Hypeâ. See for example her paper with Koller âClimbing towards NLUâ for criticisms of attributing some kinds of mental states to neural networks.
The paper you mentioned doesnât seem to discuss existential risk or AGI at all, so I donât see how it could represent the sort of object-level argument against existential risk that Peter is asking for.
Have a little imagination.
Suppose I am very worried that ghosts will steal things out of my closet. It seems like a perfectly object-level argument against my position to provide reasons for thinking that beliefs in paranormal activity are not scientifically respectable. This can be true even if the reasons provided do not mention ghosts.
People like Bender take themselves to be offering reasons for thinking that worries about AGI are not scientifically respectable. This can be true even if the reasons they provide do not mention AGI.
Note that I think Benderâs arguments are bad. But I donât see what is so mysterious about them.
It seems to me that, while the form/âmeaning distinction in this paper is certainly a fascinating one if your interests tend towards philosophy of language, this has very little to say about supposed inherent limitations of language models, and does not affect forecasts of existential risk.
Ok, let me spell it out explicitly. In a section called âLarge LMs: Hype and analysis,â the linked paper says that claims that LLM can âunderstand,â âcomprehend,â and âknowâ are âgross overclaims.â The paper supports this contention by pointing to evidence that âin fact, far from doing the âreasoningâ ostensibly required to complete the tasks, [LLMs] were instead simply more effective at leveraging artifacts in the data than previous approaches.â
Here is where the imagination comes in. Imagine that you think that all mental state attributions to artificial systems are confused in exactly this way. Imagine that you think that artificial neural nets canât reason at all. Now imagine that someone tells you that we should all be very concerned that misaligned superintelligent AI systems will destroy us.
Your response to that would be something like: it is deeply confused to think that superintelligent AI systems are something we need to worry about, and the people who are worried about them simply do not understand what is going on under the hood in machine learning models. Worries about existential risk from superintelligent AI stem from the same kind of confusion as attributing understanding to existing systems: the tendency of people who are not technically literate to anthropomorphize the systems they interact with.
Is this a real position that real living intelligent people actually hold, or is it just one of the funny contrarian philosopher beliefs that some philosophers like to around with for fun?
I think this is really the position of the stochastic parrots people, yes.
I donât think itâs plausible, but I think it partly explains their relentless opposition to work on AI safety.
I think this is an actual position. Itâs the stochastic parrots argument no? Just a recent post by a cognitive scientist holds this belief.
I donât think there were any factual claims in that article from a skim; entirely just normative claims and a few rhetorical question.