I’m also introducing another hypothetical actor, “IAIA[1]”: an organization, which could range from a private nonprofit to a treaty-backed international agency, that tracks[2] transformative AI projects and takes actions to censure or shut down dangerous ones, as well as doing other things where a central, neutral body (as opposed to an AI company) can be especially useful. (Some more details on the specifics of what IAIA can do, and what sort of “power” it might have, in a footnote.[3])
Monitoring would be with permission and assistance in the case where IAIA is a private nonprofit, i.e., in this case AI companies would be voluntarily agreeing to be monitored.
There’s a wide variety of possible powers for IAIA. For most of this post, I tend to assume that it is an agency designed for flexibility and adaptiveness, not required or enabled to execute any particular formal scheme along the lines of “If verifiable event X happens, IAIA may/must take pre-specified action Y.”
Instead, IAIA’s central tool is its informal legitimacy. It has attracted top talent and expertise, and when it issues recommendations, the recommendations are well-informed, well-argued, and commonly seen as something governments should follow by default.
In the case where IAIA has official recognition from governments or international bodies, there may be various formal provisions that make it easier for governments to quickly take IAIA’s recommendations (e.g., Congressional pre-authorizations for the executive branch to act on formal IAIA recommendations).
This doesn’t point to detailed work in the space, but in “Nearcast-based ‘deployment problem’ analysis”, Karnofsky writes:
And here’s that footnote: