td;lr Aroughframework for understanding the ways AGI gets selected for, and the ways communities can constrain that selection by acting now.
Selection toward AGI
Types of feedback loops through which AGI[1] functionality is gradually selected:
Machine learning Code internally optimised given data for continued capacity to optimise for next data inputs (toward instrumental convergence).
Human tinkering Workers tinkering with code and hardware for whatever works, as new tools used for tinkering… (toward larger neural networks and robotics that somehow work).
Institutions competing Corporate uses of AI that extract more profit/social influence/resources, which are reinvested into more machinery, allowing more uses… (toward multi-polar traps).
Artificial selection Components’ external effects selected for feeding back into their continued increased existence (toward substrate-needs convergence).
These loops are mutually reinforcing, in where and at what scale, they select for AGI.[1]
1. (within hardware) and 2. (within institutions) are about internal selection.
3. (via markets/social hierarchies/infrastructure) and 3. (via environmental effects) are about external selection.
Data (laundering) pirated from original authors, and surveyed from communities and their spaces. eg. Text and images scraped from creatives. Exchanges scraped from citizens. Data from data workers areunderpaid if paid at all.
Workers (exploitation) misled to dedicate their talent, and/or eventually replaced. eg. Talented ML researchers are recruited with techutopia claims. Software engineers are hired to set up infrastructure, and then fired. A much larger number of data workers and creatives are misled to take on gigs for training of ML models.
Misuses(of AI services) lobbied for by corporations to gain profit and power. eg. General-Purpose AI is a marketing term implying that a model can and should be used for any purpose. While profitable for Big Tech, this would violate basic system-safety practices in eg. medical, automotive and industrial engineering.
Compute (pollution) polluting, energy-sucking hardware infrastructure. eg. Of the mining, production, supply, and operationchains for chipsets.
AI companies compete on, and are bottlenecked by, the availability of each. The more we can restrict the data, workers, misuses, and compute available to AI companies (and other groups), the more we restrict paths toward AGI.
This is in rough order from least to most diffuse harms:
Personal: Taking someone’s own data, for commercial ends, without consent
Institutional: Misleading employees to contribute to developing harmful products
Societal: Destabilising society with poorly designed spaghetti code
Ecological: Polluting the planetary environment
There is ambiguity across these categories. Eg. Underpaying digital freelancers and taking their unique works to train AI involve both directly personal and institution-wideharms. And while the misuse of autonomous network-centric weapons by national militaries is harmful society-wide, the data surveillance of individual citizens targeted by drones is personal.
Less diffuse harms are easier to trace and for individuals affected to act upon.
The copying of data can in principle be traced easily. If an engineer at a whistleblows and discloses harmful training datasets, they can be linked to specific authors/citizens affected.
Very diffuse harms like leakage/emission of chemicals resulting from hardware mining, production and operation are much harder to trace and link to harmed persons. Then you get into into climate-change-like coordination problems.
What follows roughly is that we can make more traction by starting at 1. That is, first help scale projects to inform people how they are personally harmed, and enable them to take targeted (legal) actions. People informed of harms directed at them are usually more motivated to act effectively to restrict those harms, than people informed of abstract distributed harms (climate change activism, notwithstanding).
Projects to restrict the increasingly diffuse harms of 2, 3, and 4 take more time to coordinate. But to constrain the selective pressures, those projects are needed too.
Eventually, such parallel actions could converge on converge on robust bans:
Data restrictions Ban input bandwidth above certain limits.
Human-in-the-loop restrictions: Ban having input, processing, and output domains be the same.
Model use restrictions: Ban multiple-domain ML models. Ban output bandwidth/intensity above certain limits.
Compute restrictions Ban compute-processing bandwidth beyond certain limits.
More thoughts from my research mentor, Forrest Landry: ”Models need to be domain restricted. Ie, only train on one domain of interaction, and then only allow one domain of input and output – preferably different ones that are NOT in any way feedback coupled to one another through any even indirect routing of just the physical universe. Ie, that any different domain indirection coupling must and can only occur through human intermediation. Ie, humans provide inputs, and the machine outputs in a different domain of action.”
Movements resisting AI
Broader movements that are acting to restrict each prerequisite:
Environmentalism Climate researchers and reporters reporting on pollution acrosssupplychains. Local residents protesting against data centers. Potentially, masslawsuits.
What are those movements resisting AI each bottlenecked by?
4 types of AGI selection, and how to constrain them
This is a quick write-up of part 2 of my VAISU talk. See also part 1, and longer post.
td;lr A rough framework for understanding the ways AGI gets selected for,
and the ways communities can constrain that selection by acting now.
Selection toward AGI
Types of feedback loops through which AGI[1] functionality is gradually selected:
Machine learning
Code internally optimised given data for continued capacity to optimise for next data inputs (toward instrumental convergence).
Human tinkering
Workers tinkering with code and hardware for whatever works, as new tools used for tinkering… (toward larger neural networks and robotics that somehow work).
Institutions competing
Corporate uses of AI that extract more profit/social influence/resources, which are reinvested into more machinery, allowing more uses… (toward multi-polar traps).
Artificial selection
Components’ external effects selected for feeding back into their continued increased existence (toward substrate-needs convergence).
These loops are mutually reinforcing, in where and at what scale, they select for AGI.[1]
1. (within hardware) and 2. (within institutions) are about internal selection.
3. (via markets/social hierarchies/infrastructure) and 3. (via environmental effects) are about external selection.
Prerequisites for AGI
Prerequisites[2] for that selection to occur:
Data (laundering)
pirated from original authors, and surveyed from communities and their spaces.
eg. Text and images scraped from creatives. Exchanges scraped from citizens.
Data from data workers are underpaid if paid at all.
Workers (exploitation)
misled to dedicate their talent, and/or eventually replaced.
eg. Talented ML researchers are recruited with tech utopia claims. Software engineers are hired to set up infrastructure, and then fired. A much larger number of data workers and creatives are misled to take on gigs for training of ML models.
Misuses (of AI services)
lobbied for by corporations to gain profit and power.
eg. General-Purpose AI is a marketing term implying that a model can and should be used for any purpose. While profitable for Big Tech, this would violate basic system-safety practices in eg. medical, automotive and industrial engineering.
Compute (pollution)
polluting, energy-sucking hardware infrastructure.
eg. Of the mining, production, supply, and operation chains for chipsets.
AI companies compete on, and are bottlenecked by, the availability of each.
The more we can restrict the data, workers, misuses, and compute available to AI companies (and other groups), the more we restrict paths toward AGI.
This is in rough order from least to most diffuse harms:
Personal: Taking someone’s own data, for commercial ends, without consent
Institutional: Misleading employees to contribute to developing harmful products
Societal: Destabilising society with poorly designed spaghetti code
Ecological: Polluting the planetary environment
There is ambiguity across these categories.
Eg. Underpaying digital freelancers and taking their unique works to train AI involve both directly personal and institution-wide harms. And while the misuse of autonomous network-centric weapons by national militaries is harmful society-wide, the data surveillance of individual citizens targeted by drones is personal.
Less diffuse harms are easier to trace and for individuals affected to act upon.
The copying of data can in principle be traced easily. If an engineer at a whistleblows and discloses harmful training datasets, they can be linked to specific authors/citizens affected.
Very diffuse harms like leakage/emission of chemicals resulting from hardware mining, production and operation are much harder to trace and link to harmed persons. Then you get into into climate-change-like coordination problems.
What follows roughly is that we can make more traction by starting at 1.
That is, first help scale projects to inform people how they are personally harmed, and enable them to take targeted (legal) actions. People informed of harms directed at them are usually more motivated to act effectively to restrict those harms, than people informed of abstract distributed harms (climate change activism, notwithstanding).
Projects to restrict the increasingly diffuse harms of 2, 3, and 4 take more time to coordinate. But to constrain the selective pressures, those projects are needed too.
Eventually, such parallel actions could converge on converge on robust bans:
Data restrictions
Ban input bandwidth above certain limits.
Human-in-the-loop restrictions:
Ban having input, processing, and output domains be the same.
Model use restrictions:
Ban multiple-domain ML models.
Ban output bandwidth/intensity above certain limits.
Compute restrictions
Ban compute-processing bandwidth beyond certain limits.
More thoughts from my research mentor, Forrest Landry:
”Models need to be domain restricted. Ie, only train on one domain of interaction, and then only allow one domain of input and output – preferably different ones that are NOT in any way feedback coupled to one another through any even indirect routing of just the physical universe. Ie, that any different domain indirection coupling must and can only occur through human intermediation. Ie, humans provide inputs, and the machine outputs in a different domain of action.”
Movements resisting AI
Broader movements that are acting to restrict each prerequisite:
Data rights
Data auditors uncovering racially biased, copyrighted, pornographic materials.
Privacy regulators restricting/suing OpenAI, Google, Facebook, and Amazon.
Creatives and privacy orgs lobbying, and suing OpenAI, Google, and Facebook.
Data workers organising and filing lawsuits.
…
Employee activism
Workers unionising and organising strikes against AI replacing their jobs.
AI researchers quitting, and potentially whistleblowing.
…
System safety
AI ethics researchers advocating against shoddy harmful design practices.
Interpretability researchers testing for misusable deceptive functionality.
Adversarial and cybersecurity researchers reporting adversarial backdoors.
Safety engineers advising on porting over auditing practices to an “AI FDA”.
Military vets and journalists investigating corruption and the Kill Cloud.
...
Environmentalism
Climate researchers and reporters reporting on pollution across supply chains.
Local residents protesting against data centers.
Potentially, mass lawsuits.
What are those movements resisting AI each bottlenecked by?
Funding.
No-strings attached.
A more precise way of redefining AGI is ‘self-sufficient learning machinery’.
The feedback loops through which AGI functionality is selected do not correspond one-on-one with the prerequisites.