For Rob Long’s work on consciousness, I can see potential value in the research he proposes, but it’s unclear to me what the story is for its value to longtermism or x-risk specifically: it feels like the question of whether current or near-term models have consciousness is a relatively small part of the question of whether models will eventually gain consciousness, which seems like the important question for the long term, and moreover I don’t expect whether any AI system is conscious to be directly relevant to whether it poses an extinction risk.
Curious whether there’s an argument for relevance that’s more indirect or that I just didn’t think of, or whether it’s just that the LTFF will sometimes make grants even if they’re not primarily about x-risk or improving the long term future.
(I don’t think it’s bad to fund this work overall, and I’m asking this primarily to get more insight into how the LTFF thinks about their own scope.)
[not on the LTFF and also not speaking for Open Phil, just giving a personal take]
A few reactions:
Which AI systems are conscious seems like a good candidate for an extremely important humanity will need to solve at some point.
studying current systems, especially in terms of seeing what philosophical theories of consciousness have to say about them, seems like a reasonable bet to make right now if you’re excited about this problem being solved at some point.
To the extent you want to bet on a person to help push this field forward, Rob seems like an excellent bet.
On why the question seems important:
Being wrong about which systems are conscious seems dangerous in both directions:
Falsely believing systems are conscious could lead to enormous waste (trying to improve the well-being of such systems).
AI systems advocating for their own rights is a plausible way by which they could gain influence over some humans, given that at least one case of this has already happened.
Not treating genuinely conscious systems as being worthy of moral consideration on the other hand, seems like a good candidate for causing a moral catastrophe of potentially astronomical scale.
For Rob Long’s work on consciousness, I can see potential value in the research he proposes, but it’s unclear to me what the story is for its value to longtermism or x-risk specifically: it feels like the question of whether current or near-term models have consciousness is a relatively small part of the question of whether models will eventually gain consciousness, which seems like the important question for the long term, and moreover I don’t expect whether any AI system is conscious to be directly relevant to whether it poses an extinction risk.
Curious whether there’s an argument for relevance that’s more indirect or that I just didn’t think of, or whether it’s just that the LTFF will sometimes make grants even if they’re not primarily about x-risk or improving the long term future.
(I don’t think it’s bad to fund this work overall, and I’m asking this primarily to get more insight into how the LTFF thinks about their own scope.)
[not on the LTFF and also not speaking for Open Phil, just giving a personal take]
A few reactions:
Which AI systems are conscious seems like a good candidate for an extremely important humanity will need to solve at some point.
studying current systems, especially in terms of seeing what philosophical theories of consciousness have to say about them, seems like a reasonable bet to make right now if you’re excited about this problem being solved at some point.
To the extent you want to bet on a person to help push this field forward, Rob seems like an excellent bet.
On why the question seems important:
Being wrong about which systems are conscious seems dangerous in both directions:
Falsely believing systems are conscious could lead to enormous waste (trying to improve the well-being of such systems).
AI systems advocating for their own rights is a plausible way by which they could gain influence over some humans, given that at least one case of this has already happened.
Not treating genuinely conscious systems as being worthy of moral consideration on the other hand, seems like a good candidate for causing a moral catastrophe of potentially astronomical scale.