I think raw intelligence, while important, is not the primary factor that explains why humanity-as-a-species is much more powerful than chimpanzees-as-a-species. Notably, humans were once much less powerful, in our hunter-gatherer days, but over time, through the gradual process of accumulating technology, knowledge, and culture, humans now possess vast productive capacities that far outstrip our ancient powers.
...
There are strong pressures—including the principle of comparative advantage, diseconomies of scale, and gains from specialization—that incentivize making economic services narrow and modular, rather than general and all-encompassing. Illustratively, a large factory where each worker specializes in their particular role will be much more productive than a factory in which each worker is trained to be a generalist, even though no one understands any particular component of the production process very well.
What is true in human economics will apply to AI services as well. This implies we should expect something like Eric Drexler’s AI perspective, which emphasizes economic production across many agents who trade and produce narrow services, as opposed to monolithic agents that command and control.
And I updated to faster and sooner timelines from a combination of 1) noticing some potential quick improvements in AI capabilities and feeling like there could be more similar stuff in this direction, plus 2) having heard several people say they think AI is soon because (I’m inferring their “because” here) they think the innovation frontier is fructiferous (Eliezer, and conversation+tweets from Max). I am likely forgetting some people here.
While I read the Sequences in 2014-15, I did feel like the model for unfriendly AI made sense, but I was mostly deferring to Eliezer on it because I had noticed how much smarter he was than me on these things.
First of all, thank you for reporting who you’ve deferred to in different ways in specific terms. Second, thank you for putting in some extra effort to not only name this or that person but make your reasoning more transparent and legible.
I respect Matthew because when I read what he writes, it feels like I tend to agree with half of the points he makes and disagree with the other half. It’s what makes him interesting. While some are more real and some are only perceived, there are barriers to posting on the EA Forum, like an expectation of too high a burden of rigour, that have people post on social media or other forums off of this one when they can’t resist the urge to express a novel viewpoint to advance progress in EA. Matthew is one of the people I think of when I wish a lot of insightful people were more willing to post on the EA Forum.
I don’t agree with all of what you’ve presented from Matthew here or what you’ve said yourself. I might come back to specify which parts I agree and disagree with later when I’ve got more time. Right now, though, I just want to positively reinforce your writing a comment that is more like the kind of feedback from others I’d like to see more of in EA.
On timelines, other people I’ve most recently updated most on:
Matthew Barnett (I updated to slower timelines):
And I updated to faster and sooner timelines from a combination of 1) noticing some potential quick improvements in AI capabilities and feeling like there could be more similar stuff in this direction, plus 2) having heard several people say they think AI is soon because (I’m inferring their “because” here) they think the innovation frontier is fructiferous (Eliezer, and conversation+tweets from Max). I am likely forgetting some people here.
While I read the Sequences in 2014-15, I did feel like the model for unfriendly AI made sense, but I was mostly deferring to Eliezer on it because I had noticed how much smarter he was than me on these things.
First of all, thank you for reporting who you’ve deferred to in different ways in specific terms. Second, thank you for putting in some extra effort to not only name this or that person but make your reasoning more transparent and legible.
I respect Matthew because when I read what he writes, it feels like I tend to agree with half of the points he makes and disagree with the other half. It’s what makes him interesting. While some are more real and some are only perceived, there are barriers to posting on the EA Forum, like an expectation of too high a burden of rigour, that have people post on social media or other forums off of this one when they can’t resist the urge to express a novel viewpoint to advance progress in EA. Matthew is one of the people I think of when I wish a lot of insightful people were more willing to post on the EA Forum.
I don’t agree with all of what you’ve presented from Matthew here or what you’ve said yourself. I might come back to specify which parts I agree and disagree with later when I’ve got more time. Right now, though, I just want to positively reinforce your writing a comment that is more like the kind of feedback from others I’d like to see more of in EA.