Interesting and hopefully very hypothetical question. :’)
Hmm, hard to say what an AI mediated existential catastrophe within 2 years might look like, that‘s so fast. Maybe a giant leap towards a vastly superintelligent system that is so unconstrained and given so much access that it very quickly overpowers any attempt to constrain it? Feels kinda like it requires surprising carelessness… Or maybe a more narrow system that is deliberatly used to cause an existential catastrophe?
Meta
Ask people at FHI/MIRI/GovAI/OpenPhil/… what I should and shouldn’t do.
Confidentially talk to other rich and influential and trustworthy people to coordinate a joint effort
Some concrete not-well-thought-through ideas
talk to potential AI breakthrough companies about my worries, and what I can do to slow down the pace and increase safety/alignment/testing/cooperation with Safety organizations
if there are race dynamics, try anything to get some cooperation between racing parties going (probably they will already have tried that, I suppose… but maybe there will be possibilities)
Increase the number of safety researchers:
ask AI Safety researchers that are in broad agreement with you to write out big research prizes for small projects/answers to questions that might be helpful (with short time horizon, maybe 3 months, and make prizes widely known among CS people)
try to hire everyone who contributed solid work to work full time on Safety Research in a research institute you set up, give them all info you have and let them think and work on whatever makes sense… hire the best science managers you can find
offer the potential AI breakthrough companies to support them on safety issues with your hopefully somewhat impressive group of hires who won the prizes
Use of pressure/involve government:
especially unsure: if they decline for reasons that seem irresponsible, talk to the government and try to convince them that AI research is on the brink of developing huge catastrophe causing AI?
Interesting and hopefully very hypothetical question. :’)
Hmm, hard to say what an AI mediated existential catastrophe within 2 years might look like, that‘s so fast. Maybe a giant leap towards a vastly superintelligent system that is so unconstrained and given so much access that it very quickly overpowers any attempt to constrain it? Feels kinda like it requires surprising carelessness… Or maybe a more narrow system that is deliberatly used to cause an existential catastrophe?
Meta
Ask people at FHI/MIRI/GovAI/OpenPhil/… what I should and shouldn’t do.
Confidentially talk to other rich and influential and trustworthy people to coordinate a joint effort
Some concrete not-well-thought-through ideas
talk to potential AI breakthrough companies about my worries, and what I can do to slow down the pace and increase safety/alignment/testing/cooperation with Safety organizations
if there are race dynamics, try anything to get some cooperation between racing parties going (probably they will already have tried that, I suppose… but maybe there will be possibilities)
Increase the number of safety researchers:
ask AI Safety researchers that are in broad agreement with you to write out big research prizes for small projects/answers to questions that might be helpful (with short time horizon, maybe 3 months, and make prizes widely known among CS people)
try to hire everyone who contributed solid work to work full time on Safety Research in a research institute you set up, give them all info you have and let them think and work on whatever makes sense… hire the best science managers you can find
offer the potential AI breakthrough companies to support them on safety issues with your hopefully somewhat impressive group of hires who won the prizes
Use of pressure/involve government:
especially unsure: if they decline for reasons that seem irresponsible, talk to the government and try to convince them that AI research is on the brink of developing huge catastrophe causing AI?