The history of the Fair Trade movement—while very different to AI Safety in a whole bunch of ways—provides weak additional evidence in favour of trying to heavily restrict (and closely monitoring) the say that AI labs themselves get in safety-related decision-making.
I say this because involvement of major companies in Fair Trade seems to have led to a weakening of existing standards and a proliferation of (weak) certification schemes.
(Note, my overall guess is that the labs very much need to be involved in safety-related decision-making and it would be counterproductive to try to shut them out entirely. But at the very least, this should be a warning sign that you need to take proactive steps to prevent co-option.)
Here are some other relevant “strategic implications” from my report on the movement—there’s detail on the reasoning and evidence for each in the report itself.
Engaging directly with mainstream market institutions and dynamics will enable a social movement to influence consumer behavior much more rapidly than efforts to build “alternative” supply chains.
Engaging directly with mainstream market institutions and dynamics may lead to co-option and a lowering of standards.
Social movements should implement strategies to resist pressure from private sector businesses to weaken the standards of certified products.
Social movements should seek to minimize the number of competing certification schemes.
Relative, flexible certifications may be especially susceptible to downward pressure.
International standards may be seen as more credible than local or national standards.
Note, I think the analogy might hold if we replace “mainstream market institutions” with “leading for-profit companies” or some such.
The history of the Fair Trade movement—while very different to AI Safety in a whole bunch of ways—provides weak additional evidence in favour of trying to heavily restrict (and closely monitoring) the say that AI labs themselves get in safety-related decision-making.
I say this because involvement of major companies in Fair Trade seems to have led to a weakening of existing standards and a proliferation of (weak) certification schemes.
(Note, my overall guess is that the labs very much need to be involved in safety-related decision-making and it would be counterproductive to try to shut them out entirely. But at the very least, this should be a warning sign that you need to take proactive steps to prevent co-option.)
Here are some other relevant “strategic implications” from my report on the movement—there’s detail on the reasoning and evidence for each in the report itself.
Engaging directly with mainstream market institutions and dynamics will enable a social movement to influence consumer behavior much more rapidly than efforts to build “alternative” supply chains.
Engaging directly with mainstream market institutions and dynamics may lead to co-option and a lowering of standards.
Social movements should implement strategies to resist pressure from private sector businesses to weaken the standards of certified products.
Social movements should seek to minimize the number of competing certification schemes.
Relative, flexible certifications may be especially susceptible to downward pressure.
International standards may be seen as more credible than local or national standards.
Note, I think the analogy might hold if we replace “mainstream market institutions” with “leading for-profit companies” or some such.