Interesting post! In general, I think the field of computer security has lots of good examples of adversarial setups in which the party that can throw the most intelligence at a problem wins.
Probably not central to your main points, but on this:
If we assume no qualitative capability leaps from a regular human to a skilled human hacker, then we can say something like ‘X humans with regular hacking capability will be able to replace one skilled human hacker’. The effect of this on the attack workload is to increase it—i.e, we’d need a more capable system (higher N,V, lower F) to execute the attack in the same calendar time. By ignoring the skilled hacker v.s regular human distinction, the analysis here considers a worst case scenario world.
I think there’s at least one thing you’re overlooking: there is a lot of variance in human labor, and hiring well to end up on the right side of that variance is really hard. 10x engineers are real, and so are 0x and −5x engineers and −50x managers, and if you’re not careful when building your team, you’ll end up paying for 10,000 “skilled” labor hours which don’t actually accomplish much.
An AI comprised of a bunch of subagents might have vaguely similar problems if you squint, but my guess is that the ability to hire and fire relatively instantaneously, clone your most productive workers, etc. makes a pretty big difference. At the very least, the variance is probably much lower.
Another reason that I suspect 10,000 labor hours is on the high end for humans: practical offensive cybersecurity isn’t exactly the most prestigious career track. My guess is that the most cognitively-demanding offensive cybersecurity work is currently done in academia and goes into producing research papers and proofs-of-concept. Among humans, the money, prestige, and lifestyle offered by a career with a government agency or a criminal enterprise just can’t compete with the other options available in academia and industry to the best and brightest minds.
Yep, the variance of human worker teams should definitely be stressed. It’s plausible that a super team of hackers might have attack workloads on the scale of 100s to 1000s of hours [1], whereas for lower quality teams, this may be more like 100,000s of hours.
Thinking about it, I can probably see significant variance amongst AI systems due to various degrees of finetuning on cyber capabilities [2](though as you said, not as much variance as human teams). E.g: A capable foundational model may map to something like a 60th percentile hacker and so have attack workloads on the order of 10,000s of hours (like in this piece). A finetuned model might map to a 95th percentile hacker and so a team of these may have workloads on the scale of 1000s of hours.
Interesting post! In general, I think the field of computer security has lots of good examples of adversarial setups in which the party that can throw the most intelligence at a problem wins.
Probably not central to your main points, but on this:
I think there’s at least one thing you’re overlooking: there is a lot of variance in human labor, and hiring well to end up on the right side of that variance is really hard. 10x engineers are real, and so are 0x and −5x engineers and −50x managers, and if you’re not careful when building your team, you’ll end up paying for 10,000 “skilled” labor hours which don’t actually accomplish much.
An AI comprised of a bunch of subagents might have vaguely similar problems if you squint, but my guess is that the ability to hire and fire relatively instantaneously, clone your most productive workers, etc. makes a pretty big difference. At the very least, the variance is probably much lower.
Another reason that I suspect 10,000 labor hours is on the high end for humans: practical offensive cybersecurity isn’t exactly the most prestigious career track. My guess is that the most cognitively-demanding offensive cybersecurity work is currently done in academia and goes into producing research papers and proofs-of-concept. Among humans, the money, prestige, and lifestyle offered by a career with a government agency or a criminal enterprise just can’t compete with the other options available in academia and industry to the best and brightest minds.
Yep, the variance of human worker teams should definitely be stressed. It’s plausible that a super team of hackers might have attack workloads on the scale of 100s to 1000s of hours [1], whereas for lower quality teams, this may be more like 100,000s of hours.
Thinking about it, I can probably see significant variance amongst AI systems due to various degrees of finetuning on cyber capabilities [2](though as you said, not as much variance as human teams). E.g: A capable foundational model may map to something like a 60th percentile hacker and so have attack workloads on the order of 10,000s of hours (like in this piece). A finetuned model might map to a 95th percentile hacker and so a team of these may have workloads on the scale of 1000s of hours.
Though 100s of hours seems more on the implausible side—I’m guessing this would require a very large team (100s) of very skilled hackers.
And other relevant skills, like management