This is a bit of a hot-take, but I’m somewhat skeptical of the ability of standards to effectively regulate TAI. I suspect that in order to be safe, an actor will have to be willing to take measures beyond any standards, in which case implementing paragraph 23 subsection d will only be a distraction. On the other hand, standards could very easily slow the most responsible actors and cause one of the least responsible actors who doesn’t care about them at all to win the AGI race.
I can respond to your message right now via a myriad of potential software because of the establishment of a technical standard, HTTP. Additionally, all major web browsers run and interpret Javascript, in large part due to SSOs like IETF and W3C. By contrast, on mobile, we have two languages for the duopoly, and a myriad of issues I won’t go into, but suffice to say there has been a failure of SSOs in the space to replicate what happened with web browsing and early internet. It may be that TAI present novel and harder challenges, but in some of the hardest such technical coordination challenges to date, SSOs have been very useful. I’m not as worried about defection as you if we get something good going—the leaders will likely have significant resources, and therefore be under bigger public scrutiny and will want to show they are also leading on participating in standard setting. I am hopeful that there will be significant innovation in this area in the next few years. [Disclaimer, I work in this area, so naturally biased]
I guess the success of those standards for the web doesn’t feel very relevant to the problem of aligning AI. For a start, the design of the protocols has led to countless security flaws, hardly seems robust?
In addition, the technology has often evolved by messing up and then being patched later.
AI doesn’t exist in a vacuum, and TAI won’t either. AI has messed up, is messing up and will mess up bigger as it gets more advanced. Security will never be a 100% solved problem, and aiming for zero breaches of all AI systems is unrealistic. I think we’re more likely to have better AI security with standards—do you disagree with that? I’m not a security expert, but here some relevant considerations of one applied to TAI. See in particular the section “Assurance Requires Formal Proofs, Which Are Provably Impossible”. Given the probably impossible nature of having formal guarantees (not to say we shouldn’t try to get as close as possible), it really does seem that leveraging whatever institutional and coordination mechanisms have worked in the past is a worthwhile idea. I consider SSOs to be one set of these, all things considered.
Here is a section from an article written by someone who has worked in SSOs and security for decades: > Most modern encryption is based on standardised algorithms and protocols; the use of open, well-tested and thoroughly analysed encryption standards is generally recommended. WhatsApp, Facebook Messenger, Skype, and Google Messages now all use the same encryption standard (the Signal protocol) because it has proven to be secure and reliable. Even if weaknesses are found in such encryption standards, solutions are often quickly made available thanks to the sheer number of adopters.
I think that’s a valid worry and I also don’t expect the standards to end up specifying how to solve the alignment problem. :P I’d still be pretty happy about the proposed efforts on standard setting because I also expect standards to have massive effects that can be more or less useful for a) directing research in directions that reduce longterm risks (e.g. pushing for more mechanistic interpretability), b) limiting how quickly an agentic AI can escape our control (e.g. via regulating internet access, making manipulation harder), c) enabling strong(er) international agreements (e.g. shared standards could become basis for international monitoring efforts of AI development and deployment).
This is a bit of a hot-take, but I’m somewhat skeptical of the ability of standards to effectively regulate TAI. I suspect that in order to be safe, an actor will have to be willing to take measures beyond any standards, in which case implementing paragraph 23 subsection d will only be a distraction. On the other hand, standards could very easily slow the most responsible actors and cause one of the least responsible actors who doesn’t care about them at all to win the AGI race.
I can respond to your message right now via a myriad of potential software because of the establishment of a technical standard, HTTP. Additionally, all major web browsers run and interpret Javascript, in large part due to SSOs like IETF and W3C. By contrast, on mobile, we have two languages for the duopoly, and a myriad of issues I won’t go into, but suffice to say there has been a failure of SSOs in the space to replicate what happened with web browsing and early internet. It may be that TAI present novel and harder challenges, but in some of the hardest such technical coordination challenges to date, SSOs have been very useful. I’m not as worried about defection as you if we get something good going—the leaders will likely have significant resources, and therefore be under bigger public scrutiny and will want to show they are also leading on participating in standard setting. I am hopeful that there will be significant innovation in this area in the next few years. [Disclaimer, I work in this area, so naturally biased]
I guess the success of those standards for the web doesn’t feel very relevant to the problem of aligning AI. For a start, the design of the protocols has led to countless security flaws, hardly seems robust?
In addition, the technology has often evolved by messing up and then being patched later.
AI doesn’t exist in a vacuum, and TAI won’t either. AI has messed up, is messing up and will mess up bigger as it gets more advanced. Security will never be a 100% solved problem, and aiming for zero breaches of all AI systems is unrealistic. I think we’re more likely to have better AI security with standards—do you disagree with that? I’m not a security expert, but here some relevant considerations of one applied to TAI. See in particular the section “Assurance Requires Formal Proofs, Which Are Provably Impossible”. Given the probably impossible nature of having formal guarantees (not to say we shouldn’t try to get as close as possible), it really does seem that leveraging whatever institutional and coordination mechanisms have worked in the past is a worthwhile idea. I consider SSOs to be one set of these, all things considered.
Here is a section from an article written by someone who has worked in SSOs and security for decades:
> Most modern encryption is based on standardised algorithms and protocols; the use of open, well-tested and thoroughly analysed encryption standards is generally recommended. WhatsApp, Facebook Messenger, Skype, and Google Messages now all use the same encryption standard (the Signal protocol) because it has proven to be secure and reliable. Even if weaknesses are found in such encryption standards, solutions are often quickly made available thanks to the sheer number of adopters.
Standards can help with security b/c that’s more of a standard problem, but I suspect it’ll be a distraction for aligning AGI.
Well I disagree but there’s no need to agree—diverse approaches to a hard problem sounds good to me.
I think that’s a valid worry and I also don’t expect the standards to end up specifying how to solve the alignment problem. :P I’d still be pretty happy about the proposed efforts on standard setting because I also expect standards to have massive effects that can be more or less useful for
a) directing research in directions that reduce longterm risks (e.g. pushing for more mechanistic interpretability),
b) limiting how quickly an agentic AI can escape our control (e.g. via regulating internet access, making manipulation harder),
c) enabling strong(er) international agreements (e.g. shared standards could become basis for international monitoring efforts of AI development and deployment).