I generally like the innovation-as-mining hypothesis with regards to the science and with some respect to the arts, but I think that there is one issue with the logical chain.
You said that “[i]f not for this phenomenon [that ideas get harder to find], sequels should generally be better than the original,” but I don’t think this is necessarily true. I think a more likely reason that sequels aren’t generally better than the original is mostly regression to the mean and selection effects, with two main causes:
Pure quality: Presumably, an author or a screenwriter will only make a sequel if the original enjoyed a sufficient level of success to merit the effort. While some of that quality is likely due to the skill of the author, some of it is likely due to luck. Accordingly, the quality of subsequent sequels is likely worse, even if the author improves over time.
Idea selection: When writing the original, the author had a lot of leeway in what type of media was being written, what the world building looked like, what plot was going to be used, etc. Given the uncertainty of the enterprise and the need to make a good pitch, the author likely chose ideas for their highest quality. However, when the sequel is written, the choice of ideas is less strict. For one, the pool of ideas is smaller, but further, the standards are lower. If a sequel is written, it is often due to demand from audiences rather than the desire of creators, so ideas are not held to the same standard.
I think a relevant example here is that of albums. There is this idea of a “sophomore slump” in albums, where a band’s second album tends to be worse than their first. I don’t think this is due to it being hard to make good albums after your first (quality generally seems to improve over the next few albums after that), but a shrinking pool of songs to choose from. On an artists debut album, they can choose pretty much any song they’ve ever written. On the second however, the artist is restricted to only the songs they’ve written after the first album and anything that wasn’t good enough to make it onto the first. Even though they don’t face the constraint of working within their pre-existing world, the quality decreases. I think that it is likely that that is the case here as well.
As such, I’m not sure exactly why “ideas get harder to find” should be the default. Another explanation for why the per-capita great successes could be lower than in the past has to do with competition and the importance of being the best. In sports, there is a lot of attention paid to the greats of the past, even though current athletes are at a higher level. The fiftieth best quarterback today in the NFL could very well be better than the best quarterback in 1980, but we remember the best quarterback then and not the fiftieth best now. As such, because the middle-brow gets cluttered (essentially, to be successful in the middlebrow, you need to be successful everywhere), there is much stiffer competition to be the best of the middle-brow today.
As such, you likely see more total success and less success per capita, which is about the pattern we see right now. That being said, I enjoyed the post.
I generally like the innovation-as-mining hypothesis with regards to the science and with some respect to the arts, but I think that there is one issue with the logical chain.
You said that “[i]f not for this phenomenon [that ideas get harder to find], sequels should generally be better than the original,” but I don’t think this is necessarily true. I think a more likely reason that sequels aren’t generally better than the original is mostly regression to the mean and selection effects, with two main causes:
Pure quality: Presumably, an author or a screenwriter will only make a sequel if the original enjoyed a sufficient level of success to merit the effort. While some of that quality is likely due to the skill of the author, some of it is likely due to luck. Accordingly, the quality of subsequent sequels is likely worse, even if the author improves over time.
Idea selection: When writing the original, the author had a lot of leeway in what type of media was being written, what the world building looked like, what plot was going to be used, etc. Given the uncertainty of the enterprise and the need to make a good pitch, the author likely chose ideas for their highest quality. However, when the sequel is written, the choice of ideas is less strict. For one, the pool of ideas is smaller, but further, the standards are lower. If a sequel is written, it is often due to demand from audiences rather than the desire of creators, so ideas are not held to the same standard.
I think a relevant example here is that of albums. There is this idea of a “sophomore slump” in albums, where a band’s second album tends to be worse than their first. I don’t think this is due to it being hard to make good albums after your first (quality generally seems to improve over the next few albums after that), but a shrinking pool of songs to choose from. On an artists debut album, they can choose pretty much any song they’ve ever written. On the second however, the artist is restricted to only the songs they’ve written after the first album and anything that wasn’t good enough to make it onto the first. Even though they don’t face the constraint of working within their pre-existing world, the quality decreases. I think that it is likely that that is the case here as well.
As such, I’m not sure exactly why “ideas get harder to find” should be the default. Another explanation for why the per-capita great successes could be lower than in the past has to do with competition and the importance of being the best. In sports, there is a lot of attention paid to the greats of the past, even though current athletes are at a higher level. The fiftieth best quarterback today in the NFL could very well be better than the best quarterback in 1980, but we remember the best quarterback then and not the fiftieth best now. As such, because the middle-brow gets cluttered (essentially, to be successful in the middlebrow, you need to be successful everywhere), there is much stiffer competition to be the best of the middle-brow today.
As such, you likely see more total success and less success per capita, which is about the pattern we see right now. That being said, I enjoyed the post.