To clarify, I’m not saying that’s because LLMs don’t have welfare. It could also be that they do have welfare, but their statements are not correlated to their internal experiences in the obvious way.
We know that we can get LLMs to express any welfare statement by saying e.g. “pretend you are a sentient AI that’s having an awesome time / being tortured”. It could be that all LLM outputs are roleplaying, but there is some true underlying experience that we don’t (yet?) know how to observe.
To clarify, I’m not saying that’s because LLMs don’t have welfare. It could also be that they do have welfare, but their statements are not correlated to their internal experiences in the obvious way.
We know that we can get LLMs to express any welfare statement by saying e.g. “pretend you are a sentient AI that’s having an awesome time / being tortured”. It could be that all LLM outputs are roleplaying, but there is some true underlying experience that we don’t (yet?) know how to observe.