Thanks for this. If easy, can you please curate your suggested questions in a spreadsheet so that I can filter them by priority and type? If you do this, I will share with at least two academics and labs who might do some of the research desired. I may do so anyway, but at the moment, it probably won’t be something that they will find time to read unless I can refer them to the parts that are most immediately relevant.
Here is what I eventually extracted and will share, just in case it’s useful.
**★★★ (RP DG) By what year will at least 15% of patents granted in the US be for designs generated primarily via AI? Reasons for inclusion: both an early sign that AI might be able to design dangerous technology and an indicator that AIs will be economically useful to deploy across diverse industries. Question resolves according to the best estimate by the [resolution council].
**★★★ (UF RP) How long will be the gap between the first creation of an AI which could automate 65% of current labour and the availability of an equivalently capable model as a free open-source program?
**★★★ (RP) Meta-capabilities question: by 2029 will there be a better way to assess the capabilities of models than testing their performance on question-and-answer benchmarks?
**★★★ (RP UF) How much money will the Chinese government cumulatively spend on training AI models between 2024 and 2040 as estimated by the [resolution council]?
**★★★ (UF, FE, RP) Consider the first AI model able to individually perform any cognitive labour that a human can. Then, how likely is the chance of an deliberately engineered pandemic which kills >20% of the world’s population in the 50 years after the first such model is built?
**★★★ (UF, FE, RP) How does the probability of the previous question change if models are widely available to citizens and private businesses, compared to if only government and specified trusted private organizations are allowed to use them?
**★★★ (FE, RP) What is the total number of EAs in technical AI alignment Across academia, industry, independent research organizations, ¿government?, etc. See The academic contribution to AI safety seems large for an estimate from 2020.
**★★★ (FE, RP) What is the total number of non-EAs in technical AI alignment? Across academia, industry, independent research organizations, ¿government?, etc.
**★★★ (RP) How likely is it that an AI could get nanomachines built just by making ordinary commercial purchases online, and obtaining the cooperation of <30 human beings without scientific skills above masters degrees in relevant subjects?
**★★★ (UF, RP) Take-off speed: after automating 15% of labour, how long will it take until 60% of labour is automated? Question note: 99%+ of labour has been already been automated, since most humans don’t work on agriculture any more. This question asks about automating 15% and 60% of labour of the type done in 2023; see “recurring terms”.
**★★★ (FE, RP) How long does it take TSMC to manufacture 100k GPUs? Relevance: Not that high, but a neat Fermi estimate warm up. Might just generally be good for having good models of the world, though.
**★★★ (UF, RP) What is the % chance that by 2025/2030/35/40 an AI will persuade a human to commit a crime in order to further the AI’s purposes? If one wanted to make this question resolvable: Question resolves according to the [resolution council]‘s probability that this has happened. This would require a platform that accepts probabilistic resolutions. See also below “When will the US’ SEC accuse someone of committing securities fraud substantially aided by AI systems?”
**★★★ (RP, FE) What fraction of labour will be automated between 2023 and 2028/2035/2040/2050/2100? Question operationalization: See “recurring terms” section For a reference on an adjacent, see Phil Trammell’s Economic growth under transformative AI.
Thanks for this. If easy, can you please curate your suggested questions in a spreadsheet so that I can filter them by priority and type? If you do this, I will share with at least two academics and labs who might do some of the research desired. I may do so anyway, but at the moment, it probably won’t be something that they will find time to read unless I can refer them to the parts that are most immediately relevant.
Here is what I eventually extracted and will share, just in case it’s useful.
**★★★ (RP DG) By what year will at least 15% of patents granted in the US be for designs generated primarily via AI? Reasons for inclusion: both an early sign that AI might be able to design dangerous technology and an indicator that AIs will be economically useful to deploy across diverse industries. Question resolves according to the best estimate by the [resolution council].
**★★★ (UF RP) How long will be the gap between the first creation of an AI which could automate 65% of current labour and the availability of an equivalently capable model as a free open-source program?
**★★★ (RP) Meta-capabilities question: by 2029 will there be a better way to assess the capabilities of models than testing their performance on question-and-answer benchmarks?
**★★★ (RP UF) How much money will the Chinese government cumulatively spend on training AI models between 2024 and 2040 as estimated by the [resolution council]?
**★★★ (UF, FE, RP) Consider the first AI model able to individually perform any cognitive labour that a human can. Then, how likely is the chance of an deliberately engineered pandemic which kills >20% of the world’s population in the 50 years after the first such model is built?
**★★★ (UF, FE, RP) How does the probability of the previous question change if models are widely available to citizens and private businesses, compared to if only government and specified trusted private organizations are allowed to use them?
**★★★ (FE, RP) What is the total number of EAs in technical AI alignment Across academia, industry, independent research organizations, ¿government?, etc. See The academic contribution to AI safety seems large for an estimate from 2020.
**★★★ (FE, RP) What is the total number of non-EAs in technical AI alignment? Across academia, industry, independent research organizations, ¿government?, etc.
**★★★ (RP) How likely is it that an AI could get nanomachines built just by making ordinary commercial purchases online, and obtaining the cooperation of <30 human beings without scientific skills above masters degrees in relevant subjects?
**★★★ (UF, RP) Take-off speed: after automating 15% of labour, how long will it take until 60% of labour is automated? Question note: 99%+ of labour has been already been automated, since most humans don’t work on agriculture any more. This question asks about automating 15% and 60% of labour of the type done in 2023; see “recurring terms”.
**★★★ (FE, RP) How long does it take TSMC to manufacture 100k GPUs? Relevance: Not that high, but a neat Fermi estimate warm up. Might just generally be good for having good models of the world, though.
**★★★ (UF, RP) What is the % chance that by 2025/2030/35/40 an AI will persuade a human to commit a crime in order to further the AI’s purposes? If one wanted to make this question resolvable: Question resolves according to the [resolution council]‘s probability that this has happened. This would require a platform that accepts probabilistic resolutions. See also below “When will the US’ SEC accuse someone of committing securities fraud substantially aided by AI systems?”
**★★★ (RP, FE) What fraction of labour will be automated between 2023 and 2028/2035/2040/2050/2100? Question operationalization: See “recurring terms” section For a reference on an adjacent, see Phil Trammell’s Economic growth under transformative AI.
I have extracted top questions to here: https://github.com/NunoSempere/clarivoyance/blob/master/list/top-questions.md with the Linux command at the top of the page. Hope this is helpful enough.
Thank you.