Thanks for your thoughts. Regarding spreading my argument across 5 posts, I did this in part because I thought connected sequences of posts were encouraged?
Regarding the single quantity issue, I don’t think it is a red herring, because if there are multiple distinct quantities then the original argument for self-sustaining rapid growth becomes significantly weaker (see my responses to Flodorner and Lukas for more on this).
You say “Might the same thing be true of AI—that a few factors really do allow for drastic improvements in problem-solving across many domains? It’s not at all clear that it isn’t.” I believe we have good reason to think no such few factors exist. I would say because A) this does not seem to be how human intelligence works and B) because this does not seem to be consistent with the history of progress in AI research. Both I would say are characterised by many different functionalities or optimisations for particular tasks. Not to say there are no general principles but I think these are not as extensive as you seem to believe. However regardless of this point, I would just say that if Bostrom’s argument is to succeed I think he needs to give some persuasive reasons or evidence as to why we should think such factors exist. Its not sufficient just to argue that they might.
Connected sequences of posts are definitely encouraged, as they are sometimes the best way to present an extensive argument. However, I’d generally recommend that someone make one post over two short posts if they could reasonably fit their content into one post, because that makes discussion easier.
In this case, I think the content could have been fit into fewer posts (not just one, but fewer than five) had the organization system been a bit different, but this isn’t meant to be a strong criticism—you may well have chosen the best way to sort your arguments. The critique I’m most sure about is that your section on “the nature of intelligence” could have benefited from being broken down a bit more, with more subheadings and/or other language meant to guide readers through the argument (similarly to the way you presented Bostrom’s argument in the form of a set of premises, which was helpful).
Thanks for your thoughts. Regarding spreading my argument across 5 posts, I did this in part because I thought connected sequences of posts were encouraged?
Regarding the single quantity issue, I don’t think it is a red herring, because if there are multiple distinct quantities then the original argument for self-sustaining rapid growth becomes significantly weaker (see my responses to Flodorner and Lukas for more on this).
You say “Might the same thing be true of AI—that a few factors really do allow for drastic improvements in problem-solving across many domains? It’s not at all clear that it isn’t.” I believe we have good reason to think no such few factors exist. I would say because A) this does not seem to be how human intelligence works and B) because this does not seem to be consistent with the history of progress in AI research. Both I would say are characterised by many different functionalities or optimisations for particular tasks. Not to say there are no general principles but I think these are not as extensive as you seem to believe. However regardless of this point, I would just say that if Bostrom’s argument is to succeed I think he needs to give some persuasive reasons or evidence as to why we should think such factors exist. Its not sufficient just to argue that they might.
Connected sequences of posts are definitely encouraged, as they are sometimes the best way to present an extensive argument. However, I’d generally recommend that someone make one post over two short posts if they could reasonably fit their content into one post, because that makes discussion easier.
In this case, I think the content could have been fit into fewer posts (not just one, but fewer than five) had the organization system been a bit different, but this isn’t meant to be a strong criticism—you may well have chosen the best way to sort your arguments. The critique I’m most sure about is that your section on “the nature of intelligence” could have benefited from being broken down a bit more, with more subheadings and/or other language meant to guide readers through the argument (similarly to the way you presented Bostrom’s argument in the form of a set of premises, which was helpful).