Yudkowsky has been writing about how he’d set up a government in his story Mad Investor Chaos, which is, in many ways, a story about human coordination, and how the “obvious” way to do it is so completely different from anything Earth is doing.
This has nobody studying it AFAIK.
If you’re interested and not already reading the story:
TL;DR: I don’t think any of these suggestions will solve your problem for AI governance; Take this as “Yudkowsky has lots of unusual ideas about the topic of governance specifically”
“Governance” for any org
The main principle is “every decision needs to have a specific person tagged as responsible for it”.
There’s a long rant about how if there are 2 (or more) people “responsible” for a decision or making it “together”, then it’s as if there are zero people responsible for it.
The responsible person needs
“organizational eyes”: visibility to all the relevant information
“organizational fingers”: the ability to do the required actions
If a strange/unexpected case comes up, one can “throw an exception” to the person one step higher up in the hierarchy. For example, if I got seemingly contradicting orders, or if I’m missing information, or skill for a complicated decision, or if for some other reason I am not willing to take responsibility for what I’m going to decide.
All decisions are logged. There’s a process for reviewing the logs.
“Governance” for a country specifically
Elections:
Delegative voting
With several “levels”: citizens give their votes to delegates, who give their votes to higher-order-delegates , who vote for the people in government
Voting starts linar but at some point (around 200 votes?) becomes quodratic
A main principle driving this is that each person can talk directly to a representative who has an incentive to listen
Easy to overthrow:
There are a ton of safety measures that ensure that the citizens are incredibly able and ready to overthrow the government, and that the government knows this. For example:
There is a “oops let’s overthrow the government” festival once a year
The military can’t grow stronger than what the citizens can beat by themselves
Slow
The government is optimized for “not making mistakes” and absolutely not for “being able to move quickly and break things”. For example:
Every law needs someone from the government to officially take responsibility from it (meaning if something goes wrong with the intended consequences of the law—people complain to that person), including reading the law out loud. If nobody is willing to take that responsibility, the law is voided. If the law is too long, people will be more annoyed at having to read them.
Uses conditional prediction markets to chose policies.
Against [certain] corruption:
If you serve any role in government, you can’t serve any other role in the future. This makes sure that the “higher” elected officials can’t offer “jobs” to the lower elected officials (like the delegates) - and so the lower elected officials get their power only from voters, and not from the existing centralized government.
It is described that nobody is actually happy with their solution for governance (which I interpret as “it’s full of patches, it’s not based on an elegant mathematical proof on how ideal governance must certainly look”), but it’s the best they found so far, and they’re ready to overthrow it over the tiniest thing (like something 1% as bad as lying (my words)), or even “just in case” if they’re in a situation where they can’t know for sure that there’s no conspiracy going on.
Meta: It seems to me like Yudkowsky is mainly concerned in aligning the country’s government’s incentives, NOT in optimizing for ideal decisions, or fast ones, or anything like that. Mainly it seems like he’s concerned with the government not being able to abuse its power, or to become a monopoly that nobody can replace. I interpret his intent as “make it very easy for anyone with a better idea of how to run a government to replace the current government (if they are right), and everything will flow from there”, similarly to how capitalism doesn’t tell you how to run a company, but it does try making sure that if someone has a better idea than you—they can replace you.
Yudkowsky has been writing about how he’d set up a government in his story Mad Investor Chaos, which is, in many ways, a story about human coordination, and how the “obvious” way to do it is so completely different from anything Earth is doing.
This has nobody studying it AFAIK.
If you’re interested and not already reading the story:
I can try finding that part for you
Also see this bounty.
My imperfect memory:
TL;DR: I don’t think any of these suggestions will solve your problem for AI governance; Take this as “Yudkowsky has lots of unusual ideas about the topic of governance specifically”
“Governance” for any org
The main principle is “every decision needs to have a specific person tagged as responsible for it”.
There’s a long rant about how if there are 2 (or more) people “responsible” for a decision or making it “together”, then it’s as if there are zero people responsible for it.
The responsible person needs
“organizational eyes”: visibility to all the relevant information
“organizational fingers”: the ability to do the required actions
If a strange/unexpected case comes up, one can “throw an exception” to the person one step higher up in the hierarchy. For example, if I got seemingly contradicting orders, or if I’m missing information, or skill for a complicated decision, or if for some other reason I am not willing to take responsibility for what I’m going to decide.
All decisions are logged. There’s a process for reviewing the logs.
“Governance” for a country specifically
Elections:
Delegative voting
With several “levels”: citizens give their votes to delegates, who give their votes to higher-order-delegates , who vote for the people in government
Voting starts linar but at some point (around 200 votes?) becomes quodratic
A main principle driving this is that each person can talk directly to a representative who has an incentive to listen
Easy to overthrow:
There are a ton of safety measures that ensure that the citizens are incredibly able and ready to overthrow the government, and that the government knows this. For example:
There is a “oops let’s overthrow the government” festival once a year
The military can’t grow stronger than what the citizens can beat by themselves
Slow
The government is optimized for “not making mistakes” and absolutely not for “being able to move quickly and break things”. For example:
Every law needs someone from the government to officially take responsibility from it (meaning if something goes wrong with the intended consequences of the law—people complain to that person), including reading the law out loud. If nobody is willing to take that responsibility, the law is voided. If the law is too long, people will be more annoyed at having to read them.
Uses conditional prediction markets to chose policies.
Against [certain] corruption:
If you serve any role in government, you can’t serve any other role in the future. This makes sure that the “higher” elected officials can’t offer “jobs” to the lower elected officials (like the delegates) - and so the lower elected officials get their power only from voters, and not from the existing centralized government.
It is described that nobody is actually happy with their solution for governance (which I interpret as “it’s full of patches, it’s not based on an elegant mathematical proof on how ideal governance must certainly look”), but it’s the best they found so far, and they’re ready to overthrow it over the tiniest thing (like something 1% as bad as lying (my words)), or even “just in case” if they’re in a situation where they can’t know for sure that there’s no conspiracy going on.
See some more here.
Meta: It seems to me like Yudkowsky is mainly concerned in aligning the country’s government’s incentives, NOT in optimizing for ideal decisions, or fast ones, or anything like that. Mainly it seems like he’s concerned with the government not being able to abuse its power, or to become a monopoly that nobody can replace. I interpret his intent as “make it very easy for anyone with a better idea of how to run a government to replace the current government (if they are right), and everything will flow from there”, similarly to how capitalism doesn’t tell you how to run a company, but it does try making sure that if someone has a better idea than you—they can replace you.