Thanks, I think this is subtle and I don’t think I expressed this perfectly.
> If someone uses AI capabilities to create a synthetic virus (which they wouldn’t have been able to do in the counterfactual world without that AI-generated capability) and caused the extinction or drastic curtailment of humanity, would that count as “AGI being developed”?
No, I would not count this.
I’d probably count it if the AI a) somehow formed the intention to do this and then developed the pathogen and released it without human direction, but b) couldn’t yet produce as much economic output as full automation of labor.
Thanks, I think this is subtle and I don’t think I expressed this perfectly.
> If someone uses AI capabilities to create a synthetic virus (which they wouldn’t have been able to do in the counterfactual world without that AI-generated capability) and caused the extinction or drastic curtailment of humanity, would that count as “AGI being developed”?
No, I would not count this.
I’d probably count it if the AI a) somehow formed the intention to do this and then developed the pathogen and released it without human direction, but b) couldn’t yet produce as much economic output as full automation of labor.
Okay great, that makes sense to me. Thank you very much for the clarification!