This is a thoughtful framework, and I broadly find the approach reasonable. One dimension I’d like to see explored further, though, is the risks embedded in using collective user preference as the mechanism for determining what counts as “prosocial.”
The post rightly flags the challenge of identifying uncontroversial prosocial actions, and grounding this in aggregated user preferences is an intuitive starting point. But collective preference carries well-documented risks, including majoritarian bias, and what users collectively want may not align with what is genuinely beneficial for minority groups or for society in the long run. The history of democratic theory gives us good reason to be cautious here.
This raises a question I’d genuinely like to hear views on: to what extent should formal governance structures, including governments and their regulatory capacity, play a role in defining the boundaries of prosocial AI behaviour? I recognise the practical complexity here, particularly given the current fragmented state of AI governance globally. But philosophically, democratically accountable institutions offer something that collective user preference alone cannot: legitimacy derived from deliberative processes, legal accountability, and explicit protections for minority interests.
I’m not arguing that regulation is a clean solution. But it might serve as a useful complementary layer to user preference aggregation, providing a check against the most significant failure modes of purely preference-based approaches.
This is a thoughtful framework, and I broadly find the approach reasonable. One dimension I’d like to see explored further, though, is the risks embedded in using collective user preference as the mechanism for determining what counts as “prosocial.”
The post rightly flags the challenge of identifying uncontroversial prosocial actions, and grounding this in aggregated user preferences is an intuitive starting point. But collective preference carries well-documented risks, including majoritarian bias, and what users collectively want may not align with what is genuinely beneficial for minority groups or for society in the long run. The history of democratic theory gives us good reason to be cautious here.
This raises a question I’d genuinely like to hear views on: to what extent should formal governance structures, including governments and their regulatory capacity, play a role in defining the boundaries of prosocial AI behaviour? I recognise the practical complexity here, particularly given the current fragmented state of AI governance globally. But philosophically, democratically accountable institutions offer something that collective user preference alone cannot: legitimacy derived from deliberative processes, legal accountability, and explicit protections for minority interests.
I’m not arguing that regulation is a clean solution. But it might serve as a useful complementary layer to user preference aggregation, providing a check against the most significant failure modes of purely preference-based approaches.