A man in a hole needs a ladder, not climbing skills?
Kai Williams
Karma: 17
Thanks for the piece. I think there’s an unexamined assumption here about the robustness of non-earth settlement. It may be that one can maintain a settlement on another world for a long time, but unless we get insanely lucky, it seems unlikely to me that you live on another planet without sustaining technology at or above our current capabilities. It may also be that in the medium-term these settlements are dependent on Earth for manufacturing resources etc. which reduces their independence.
This isn’t fatal to your thesis (especially in the long-long term), but I think having a high minimum technology threshold does undercut your thesis to some extent.
I don’t have a good answer to this, but I did read a blog post recently which might be relevant. In it, two philosophers summarize their paper which argues against drawing the conclusion that longtermists should hasten extinction rather then preventing it. (The instigation of their paper was this paper by Richard Pettigrew which argued that longtermism should be highly risk-averse. I realize that this is a slightly separate question, but the discussion seems relevant.) Hope this helps!
Thanks for releasing this. I’m curious what is the more interesting sample here: somewhat established alignment researchers (measured by the proxy that they have published a paper), or the general population of who filled out the sample (including those with briefer prior engagement)?
I filled out this survey because it got signal boosted in the AI Safety Camp slack. At the same time, there were questions about the funding viability of AI Safety Camp, so I was strongly motivated to fill it out for the $40 donation. At the same time, I’m not sure that I have engaged deeply enough with Alignment Research to be called an “alignment researcher.” Given that AISC was the most common place donated to, this may have skewed the population in general.
Skimming through the visualization tool (very cool, thank you!), the personality questions didn’t seem to be affected, but the political questions do seem to vary a little bit. For instance, among alignment researchers who have published a paper, around 55% support or strongly support pausing. On the other hand, among those who haven’t published a paper, around 75% support or strongly support pausing. Which population should this analysis rely on?
I generally like the innovation-as-mining hypothesis with regards to the science and with some respect to the arts, but I think that there is one issue with the logical chain.
You said that “[i]f not for this phenomenon [that ideas get harder to find], sequels should generally be better than the original,” but I don’t think this is necessarily true. I think a more likely reason that sequels aren’t generally better than the original is mostly regression to the mean and selection effects, with two main causes:
Pure quality: Presumably, an author or a screenwriter will only make a sequel if the original enjoyed a sufficient level of success to merit the effort. While some of that quality is likely due to the skill of the author, some of it is likely due to luck. Accordingly, the quality of subsequent sequels is likely worse, even if the author improves over time.
Idea selection: When writing the original, the author had a lot of leeway in what type of media was being written, what the world building looked like, what plot was going to be used, etc. Given the uncertainty of the enterprise and the need to make a good pitch, the author likely chose ideas for their highest quality. However, when the sequel is written, the choice of ideas is less strict. For one, the pool of ideas is smaller, but further, the standards are lower. If a sequel is written, it is often due to demand from audiences rather than the desire of creators, so ideas are not held to the same standard.
I think a relevant example here is that of albums. There is this idea of a “sophomore slump” in albums, where a band’s second album tends to be worse than their first. I don’t think this is due to it being hard to make good albums after your first (quality generally seems to improve over the next few albums after that), but a shrinking pool of songs to choose from. On an artists debut album, they can choose pretty much any song they’ve ever written. On the second however, the artist is restricted to only the songs they’ve written after the first album and anything that wasn’t good enough to make it onto the first. Even though they don’t face the constraint of working within their pre-existing world, the quality decreases. I think that it is likely that that is the case here as well.
As such, I’m not sure exactly why “ideas get harder to find” should be the default. Another explanation for why the per-capita great successes could be lower than in the past has to do with competition and the importance of being the best. In sports, there is a lot of attention paid to the greats of the past, even though current athletes are at a higher level. The fiftieth best quarterback today in the NFL could very well be better than the best quarterback in 1980, but we remember the best quarterback then and not the fiftieth best now. As such, because the middle-brow gets cluttered (essentially, to be successful in the middlebrow, you need to be successful everywhere), there is much stiffer competition to be the best of the middle-brow today.
As such, you likely see more total success and less success per capita, which is about the pattern we see right now. That being said, I enjoyed the post.