This is confused, afaict? When comparing the impact of time-buying vs direct work, the probability of success for both activities is negated by the number of people pushing capabilities. So it cancels out, and you don’t need to think about the number of people in opposition.
The unique thing about time-buying is that its marginal impact increases with the number of alignment workers,[1] whereas the marginal impact of direct work plausibly decreases with the number of people already working on it (due to fruit depletion and coordination issues).[2]
If there are 300 people doing direct alignment and you’re an average worker, you can expect to contribute 0.3% of the direct work that happens per unit time. On the other hand, if you spend your time on time-buying instead, you only need to expect to save 0.3% units of time per unit of time you spend in order to break even.[3]
Although the marginal impact of every additional unit of time scales with the number of workers, there are probably still diminishing returns to more people working on time-buying.
Probably direct work scales according to some weird curve idk, but I’m guessing we’re past the peak. Two people doing direct work collaboratively do more good per person than one person. But there are probably steep diminishing marginal returns from economy-of-scale/specialisation, coordination, and motivation in this case.
Impact is a multiplication of the number of workers N, their average rate of work w, and the time they have left to work t. And because multiplication is commutative, if you increase one of the variables by a proportion r, that is equivalent to increasing any of the other variables with the same proportion. N×(w×r)×t=N×w×(t×r).
When comparing the impact of time-buying vs direct work, the probability of success for both activities is negated by the number of people pushing capabilities. So it cancels out, and you don’t need to think about the number of people in opposition.
Time-buying (slowing down AGI development) seems more directly opposed to the interests of those pushing capabilities than working on AGI safety.
If the alignment tax is low (to the tune of an open-source Python package that just lets you do “pip install alignment”) I expect all the major AGI labs to be willing to pay it. Maybe they’ll even thank you.
On the other hand, asking people to hold off on building AGI (though I agree there’s more and less clever ways to do it in practice) seems to scale poorly especially with the number of people wanting to do AGI research, and to a lesser degree the number of people doing AI/ML research in general. Or even non-researchers whose livelihoods depends on such advancements. At the very least, I do not expect effort needed to persuade people to be constant with respect to the number of people with a stake in AGI development.
Fair points. On the third hand, the more AGI researchers there are, the more “targets” there are for important arguments to reach, and the higher impact systematic AI governance interventions will have.
At this point, I seem to have lost track of my probabilities somewhere in the branches, let me try to go back and find it...
This is confused, afaict? When comparing the impact of time-buying vs direct work, the probability of success for both activities is negated by the number of people pushing capabilities. So it cancels out, and you don’t need to think about the number of people in opposition.
The unique thing about time-buying is that its marginal impact increases with the number of alignment workers,[1] whereas the marginal impact of direct work plausibly decreases with the number of people already working on it (due to fruit depletion and coordination issues).[2]
If there are 300 people doing direct alignment and you’re an average worker, you can expect to contribute 0.3% of the direct work that happens per unit time. On the other hand, if you spend your time on time-buying instead, you only need to expect to save 0.3% units of time per unit of time you spend in order to break even.[3]
Although the marginal impact of every additional unit of time scales with the number of workers, there are probably still diminishing returns to more people working on time-buying.
Probably direct work scales according to some weird curve idk, but I’m guessing we’re past the peak. Two people doing direct work collaboratively do more good per person than one person. But there are probably steep diminishing marginal returns from economy-of-scale/specialisation, coordination, and motivation in this case.
Impact is a multiplication of the number of workers N, their average rate of work w, and the time they have left to work t. And because multiplication is commutative, if you increase one of the variables by a proportion r, that is equivalent to increasing any of the other variables with the same proportion. N×(w×r)×t=N×w×(t×r).
Time-buying (slowing down AGI development) seems more directly opposed to the interests of those pushing capabilities than working on AGI safety.
If the alignment tax is low (to the tune of an open-source Python package that just lets you do “pip install alignment”) I expect all the major AGI labs to be willing to pay it. Maybe they’ll even thank you.
On the other hand, asking people to hold off on building AGI (though I agree there’s more and less clever ways to do it in practice) seems to scale poorly especially with the number of people wanting to do AGI research, and to a lesser degree the number of people doing AI/ML research in general. Or even non-researchers whose livelihoods depends on such advancements. At the very least, I do not expect effort needed to persuade people to be constant with respect to the number of people with a stake in AGI development.
Fair points. On the third hand, the more AGI researchers there are, the more “targets” there are for important arguments to reach, and the higher impact systematic AI governance interventions will have.
At this point, I seem to have lost track of my probabilities somewhere in the branches, let me try to go back and find it...
Good discussion, ty. ^^