While motivated reasoning is certainly something to look out for, the substance of the argument should also be taken into account. I believe that the main point of this post, that Yudkowsky and Soares’s book is full of narrative arguments and unfalsifiable hypotheses mostly unsupported by references to external evidence, is obviously true. As you yourself say, OP’s arguments are reasonable. On that background, this kind of attack from you seems unjustified, and I’d like to hear what parts/viewpoints/narratives/conclusions of the post are motivated reasoning in your estimation.
I do agree that motivated reasoning is common with the proponents of AI adoption. As an example, I think the white paper Sparks of Artificial General Intelligence: Early experiments with GPT-4 by Microsoft is clearly a piece of advertising masquerading as a scientific paper. Microsoft has a lot to benefit from the commercial success of its partner company OpenAI, and the conclusions it suggests are almost certainly colored by this. Same could be said about many of OpenAI’s own white papers. But this does not mean that the examples or experiments they showcase are wrong per se (even if cherry-picked), or that there is no real information in them. Their results merely need to be read with the skeptical lenses.
We should generally be skeptical of corporations (or even non-profits!) releasing pre-prints that look like scientific papers but might not pass peer review at a scientific journal. We should indeed view such pre-prints as somewhere between research and marketing. OpenAI’s pre-prints or white papers are a good example.
I think it’s hard to claim that a pre-print like Sparks of AGI is insincere (it might be, but how could we support that claim?), but this doesn’t undermine the general point. Suppose employees at Microsoft Research wanted to publish a similar report arguing that GPT-4′s seeming cognitive capabilities are actually just a bunch of cheap tricks and not sparks of anything. Would Microsoft publish that report? It’s not just about how financial or job-related incentives shape what you believe (although that is worth thinking about), it’s also about how they shape what you can say out loud. (And, importantly, what you are encouraged to focus on.)
While motivated reasoning is certainly something to look out for, the substance of the argument should also be taken into account. I believe that the main point of this post, that Yudkowsky and Soares’s book is full of narrative arguments and unfalsifiable hypotheses mostly unsupported by references to external evidence, is obviously true. As you yourself say, OP’s arguments are reasonable. On that background, this kind of attack from you seems unjustified, and I’d like to hear what parts/viewpoints/narratives/conclusions of the post are motivated reasoning in your estimation.
I do agree that motivated reasoning is common with the proponents of AI adoption. As an example, I think the white paper Sparks of Artificial General Intelligence: Early experiments with GPT-4 by Microsoft is clearly a piece of advertising masquerading as a scientific paper. Microsoft has a lot to benefit from the commercial success of its partner company OpenAI, and the conclusions it suggests are almost certainly colored by this. Same could be said about many of OpenAI’s own white papers. But this does not mean that the examples or experiments they showcase are wrong per se (even if cherry-picked), or that there is no real information in them. Their results merely need to be read with the skeptical lenses.
We should generally be skeptical of corporations (or even non-profits!) releasing pre-prints that look like scientific papers but might not pass peer review at a scientific journal. We should indeed view such pre-prints as somewhere between research and marketing. OpenAI’s pre-prints or white papers are a good example.
I think it’s hard to claim that a pre-print like Sparks of AGI is insincere (it might be, but how could we support that claim?), but this doesn’t undermine the general point. Suppose employees at Microsoft Research wanted to publish a similar report arguing that GPT-4′s seeming cognitive capabilities are actually just a bunch of cheap tricks and not sparks of anything. Would Microsoft publish that report? It’s not just about how financial or job-related incentives shape what you believe (although that is worth thinking about), it’s also about how they shape what you can say out loud. (And, importantly, what you are encouraged to focus on.)