Regarding the specific problems of specific large organizations, maybe there’s something here to do with bureaucratic mazes? For instance, Raemon’s post Recursive Middle Manager Hell has a section titled Implications for EA and AI that goes like so:
I think it is sometimes appropriate to build large organizations, when you’re trying to do a reasonably simple thing at scale.
I think most effective altruist and AI alignment organizations cannot afford to become mazes. Our key value propositions are navigating a confusing world where we don’t really know what to do, and our feedback loops are incredibly poor. We’re not sure what counts as alignment progress, many things that might help with alignment also help with AI capabilities and push us closer to either hard takeoff or a slow rolling unstoppable apocalypse.
Each stage of organizational growth triggers a bit less contact with reality, a bit more incentive to frame things so they look good.
I keep talking to people who think “Obviously, the thing we need to do is hire more. We’re struggling to get stuff done, we need more people.” And yes, you are struggling to get stuff done. But I think growing your org will diminish your ability to think, which is one of your rarest and most precious resources.
Look at the 5 example anecdotes I give, and imagine what happens, not when they are happening individually, but all at once, reinforcing each other. When managers are encouraging their researchers to think in terms of legible accomplishments. When managers are encouraging their researchers or programmers to lie. When projects acquire inertia and never stop even if they’re pointless, or actively harmful – because they look good and even a dedicated rationalist feels immense pressure to make up reasons his project is worthwhile.
Imagine if my silly IT project had been a tool or research program that turned out to be AI capabilities accelerating, and then the entire company culture converged to make that difficult to stop, or talk plainly about, or even avoid actively lying about it.
What exactly do we do about this is a bigger post. But for now: If your instinct is to grow – grow your org, or grow the effective altruism or AI safety network, think seriously about the costs of scale.
Regarding the specific problems of specific large organizations, maybe there’s something here to do with bureaucratic mazes? For instance, Raemon’s post Recursive Middle Manager Hell has a section titled Implications for EA and AI that goes like so:
I’d agree with that the above quote in that I’m not suggesting we should hire a lot more.
The way I see it, we already have a lot of people—the main choice is how to best configure them.