From my understanding of boards and governance structures, I think that few are actually very effective, and it’s often very difficult to tell this from outside the organization.
It seems valuable to differentiate between “ineffective by design” and “ineffective in practice”. Which do you think is more the cause for the trend you’re observing?
OP is concerned that Anthropic’s governance might fall into the “ineffective by design” category. Like, it’s predictable in advance that something could maybe go wrong here.
If yours is more of an “ineffective in practice” argument—that seems especially concerning, if the “ineffective in practice” point applies even when the governance appeared to be effective by design, ex ante.
In any case, I’d really like to see dedicated efforts to argue for ideal AI governance structures and documents. It feels like EA has overweighted the policy side of AI governance and underweighted the organizational founding documents side. Right now we’re in the peanut gallery, criticizing how things are going at OpenAI and now Anthropic, without offering much in the way of specific alternatives.
Events at OpenAI have shown that this issue deserves a lot more attention, in my opinion. Some ideas:
A big cash prize for best AI lab governance structure proposals. (In practice you’d probably want to pick and choose the best ideas across multiple proposals.)
Subsidize red-teaming novel proposals and testing out novel proposals in lower-stakes situations, for non-AI organiations. (All else equal, it seems better for AGI to be developed using an institutional template that’s battle-tested.) We could dogfood proposals by using them for non-AI EA startups or EA organizations focused on e.g. community-building.
Governance lit reviews to gather and summarize info, both empirical info and also theoretical models from e.g. economics. Cross-national comparisons might be especially fruitful if we don’t think the right structures are battle-tested in a US legal context.
At this point, I’m embarrassed that if someone asked me how to fix OpenAI’s governance docs, I wouldn’t really have a suggestion. On the other hand, if we had some really solid suggestions, it feels doable to either translate them into policy requirements, or convince groups like Anthropic’s trustees to adopt them.
If yours is more of an “ineffective in practice” argument—that seems especially concerning,
Good point here. You’re right that I’m highlighting “ineffective in practice”.
In any case, I’d really like to see dedicated efforts to argue for ideal AI governance structures and documents.
Yep, I’d also like to see more work here! A while back I helped work on this project, which investigated this area a bit, but I think it’s obvious there’s more work to do.
It seems valuable to differentiate between “ineffective by design” and “ineffective in practice”. Which do you think is more the cause for the trend you’re observing?
OP is concerned that Anthropic’s governance might fall into the “ineffective by design” category. Like, it’s predictable in advance that something could maybe go wrong here.
If yours is more of an “ineffective in practice” argument—that seems especially concerning, if the “ineffective in practice” point applies even when the governance appeared to be effective by design, ex ante.
In any case, I’d really like to see dedicated efforts to argue for ideal AI governance structures and documents. It feels like EA has overweighted the policy side of AI governance and underweighted the organizational founding documents side. Right now we’re in the peanut gallery, criticizing how things are going at OpenAI and now Anthropic, without offering much in the way of specific alternatives.
Events at OpenAI have shown that this issue deserves a lot more attention, in my opinion. Some ideas:
A big cash prize for best AI lab governance structure proposals. (In practice you’d probably want to pick and choose the best ideas across multiple proposals.)
Subsidize red-teaming novel proposals and testing out novel proposals in lower-stakes situations, for non-AI organiations. (All else equal, it seems better for AGI to be developed using an institutional template that’s battle-tested.) We could dogfood proposals by using them for non-AI EA startups or EA organizations focused on e.g. community-building.
Governance lit reviews to gather and summarize info, both empirical info and also theoretical models from e.g. economics. Cross-national comparisons might be especially fruitful if we don’t think the right structures are battle-tested in a US legal context.
At this point, I’m embarrassed that if someone asked me how to fix OpenAI’s governance docs, I wouldn’t really have a suggestion. On the other hand, if we had some really solid suggestions, it feels doable to either translate them into policy requirements, or convince groups like Anthropic’s trustees to adopt them.
Good point here. You’re right that I’m highlighting “ineffective in practice”.
Yep, I’d also like to see more work here! A while back I helped work on this project, which investigated this area a bit, but I think it’s obvious there’s more work to do.