I’ve heard this argument a lot (eg in the context of impact markets) and I agree that this consideration is real, but I’m not sure that it should be weighted heavily. I think it depends a lot on what the distribution of impact looks like: the size of the best positive outcomes vs the worst negative ones, their relative frequency, how different interventions (eg adding screening steps) reduces negative projects but also discourages positive ones.
For example, if in 100 projects, you have [1x +1000, 4x −100, 95x ~0], then I think black swarm farming still does a lot better than some process where you try to select the top 10 or something. Meanwhile if your outcomes look more like [2x +1000, 3x −1000, 95x ~0] then careful filtering starts to matter a lot.
My intuition is that the best projects are much better than the worst projects are bad, and also that the best projects don’t necessarily look that good at the outset. (To use the example I’m most familiar with, Manifold looked pretty sketchy when we applied for ACX Grants, and got turned down by YC and EA Bahamas; I’m still pretty impressed that Scott figured we were worth funding :P)
I’ve heard this argument a lot (eg in the context of impact markets) and I agree that this consideration is real, but I’m not sure that it should be weighted heavily. I think it depends a lot on what the distribution of impact looks like: the size of the best positive outcomes vs the worst negative ones, their relative frequency, how different interventions (eg adding screening steps) reduces negative projects but also discourages positive ones.
For example, if in 100 projects, you have [1x +1000, 4x −100, 95x ~0], then I think black swarm farming still does a lot better than some process where you try to select the top 10 or something. Meanwhile if your outcomes look more like [2x +1000, 3x −1000, 95x ~0] then careful filtering starts to matter a lot.
My intuition is that the best projects are much better than the worst projects are bad, and also that the best projects don’t necessarily look that good at the outset. (To use the example I’m most familiar with, Manifold looked pretty sketchy when we applied for ACX Grants, and got turned down by YC and EA Bahamas; I’m still pretty impressed that Scott figured we were worth funding :P)