Thank you for sharing! Great post and I’m glad there’s more attention going towards standard-setting activities. Some misc. ‘off the top of my head’ thoughts:
You’re right to highlight that standards are not a panacea and can be difficult in practice:
Looking into when and why companies deviate from standards would be a useful area of study. What would the Volkswagen emissions scandal look like for TAI?
As you mention, it might be difficult to find consensus where other stakeholders are not necessarily aligned or have the same incentives. A potentially useful thought experiment could be “if we were negotiating standards with Yann LeCun, where/why would we disagree?”
Trade secrets and intellectual property considerations are also important in this process: this piece on 5G standards and Huawei is quite illustrative. This could be either a blocker or an opportunity depending on how you see it.
Geopolitical challenges (see this and this) might make things a bit more complicated in practice.
Some SSOs work closely with others: e.g. CEN/CENELEC and ISO. I’m not familiar with their work but orgs like OCEANIS might be worth looking into as well.
Companies tend to value harmonisation, so avoiding fragmentation should be a key aim too.
In order to do standards well, a lot of that work on measurement and assessment is needed first—though a lot of this work is ongoing .
From a US policy POV, this bill might be of interest. Worth thinking about the impact of subsidizing or incentivising the involvement of more small and medium-sized companies.
I think an interesting project might be developing three ‘real’ AI use cases and assessing throughout what best practice / desirable standards might look like. It’s a complex area for AI systems in particular, so a demonstration would be very persuasive. Definitely an area where AI capabilities and AI safety people could work together on. And perhaps policymakers and regulators (like the ICO in the UK) could facilitate this with sandboxes.
Just my 2c. Very supportive otherwise, as this is definitely an under-explored area: haven’t seen much on standards in EA world since Peter Cihon’s excellent paper. Thanks afor sharing :)
Thank you for sharing! Great post and I’m glad there’s more attention going towards standard-setting activities. Some misc. ‘off the top of my head’ thoughts:
You’re right to highlight that standards are not a panacea and can be difficult in practice:
Looking into when and why companies deviate from standards would be a useful area of study. What would the Volkswagen emissions scandal look like for TAI?
As you mention, it might be difficult to find consensus where other stakeholders are not necessarily aligned or have the same incentives. A potentially useful thought experiment could be “if we were negotiating standards with Yann LeCun, where/why would we disagree?”
Trade secrets and intellectual property considerations are also important in this process: this piece on 5G standards and Huawei is quite illustrative. This could be either a blocker or an opportunity depending on how you see it.
Geopolitical challenges (see this and this) might make things a bit more complicated in practice.
Some SSOs work closely with others: e.g. CEN/CENELEC and ISO. I’m not familiar with their work but orgs like OCEANIS might be worth looking into as well.
Companies tend to value harmonisation, so avoiding fragmentation should be a key aim too.
In order to do standards well, a lot of that work on measurement and assessment is needed first—though a lot of this work is ongoing .
From a US policy POV, this bill might be of interest. Worth thinking about the impact of subsidizing or incentivising the involvement of more small and medium-sized companies.
I think an interesting project might be developing three ‘real’ AI use cases and assessing throughout what best practice / desirable standards might look like. It’s a complex area for AI systems in particular, so a demonstration would be very persuasive. Definitely an area where AI capabilities and AI safety people could work together on. And perhaps policymakers and regulators (like the ICO in the UK) could facilitate this with sandboxes.
Just my 2c. Very supportive otherwise, as this is definitely an under-explored area: haven’t seen much on standards in EA world since Peter Cihon’s excellent paper. Thanks afor sharing :)
Just to add to UK regulator stuff in the space: the DRCF has a stream on algorithm auditing. Here is a paper with a short section on standards. Obviously it’s early days, and focused on current AI systems, but it’s a start: https://www.gov.uk/government/publications/findings-from-the-drcf-algorithmic-processing-workstream-spring-2022/auditing-algorithms-the-existing-landscape-role-of-regulators-and-future-outlook