# djbinder

Karma: 65

I’m a senior research scholar at FHI, with a background in theoretical physics.

• Thanks for the reply Rory! I think at this point it is fairly clear where we agree (quantitative methods and ideas from maths and physics can be helpful in other disciplines) and where we disagree (whether complexity science has new insights to offer, and whether there is a need for an interdisciplinary field doing this work separate from the ones that already exist), and don’t have any more to offer here past my previous comments. And I appreciate your candidness noting that most complexity scientists don’t mention complexity or emergence much in their published research; as is probably clear I think this suggests that, despite their rhetoric, they haven’t managed to make these concepts useful.

I do not think the SFI, at least judging from their website, and from the book Scale which I read a few years ago, is a good model of public relations that EAs should try to emulate. They make grand claims about what they have achieved which seems to me to be out of proportion to their actual accomplishments. I’m curious to hear what you think the great success stories of SFI are. The one I know the most about, the scaling laws, I’m pretty skeptical of for the reasons outlined previously. I had a look at their “Evolution of Human Languages” program, and it seems to be fringe research by the standards of mainstream comparative linguistics. But there could well be success stories that I am unfamiliar with, particularly in economics.

• If the OP wants to discuss agent-based modeling, then I think they should discuss agent-based modeling. I don’t think there is anything to be gained by calling agent-based models “complex systems”, or that taking a complexity science viewpoint adds any value.

Likewise, if you want to study networks, why not study networks? Again, adding the word “complex” doesn’t buy you anything.

As I said in my original comment, part of complexity science is good: this is the idea we can use maths and physics to modeling other systems. But this is hardly a new insight. Economists, biophysicists, mathematical biologists, computer scientists, statisticians, and applied mathematicians have been doing this for centuries. While sometimes siloing can be a problem, for the most part ideas flow fairly freely between these disciplines and there is a lot of cross-pollination. When ideas don’t flow it is usually because they aren’t useful in the new field. (Maybe they rely on inappropriate assumptions, or are useful in the wrong regime, or answer the wrong questions, or are trivial and/​or intractable in situations the new field cares about, or don’t give empirically testable results, or are already used by the new field in a slightly different way.) The “problem” of “siloing” that complexity science claims to want to solve is largely a mirage.

But of course, complexity science makes greater claims than just this. It claims to be developing some general insights into the workings of complex systems. As I’ve noted in my previous comment, these claims are at best just false and at worst completely vacuous. I think it is dangerous to support the kind of sophistry spouted by complexity scientists, for the same reason it is dangerous to support sophistry anywhere. At best it draws attention away from scientists who are making progress on real problems, and at worst it leads to piles of misleading and overblown hype.

My criticism is not analogous to the claim that “ML is just a rebranding of statistics”. After all, ML largely studies different topics and different questions to statistics. No, it would be as if we lived in a world without computers, and ML consisted of people waxing lyrically about how “computation” would solve learning, but then when asked how would just say basic (and sometimes incorrect) things about statistics.

• As someone with a background in theoretical physics, I am very skeptical of the claims made by complexity science. At a meta-level I dislike being overly negative, and I don’t want to discourage people posting things that they think might be interesting or relevant on the forum. But I have seen complexity science discussed now by quite a few EAs rather credulously, and I think it is important to set the record straight.

On to the issues with complexity science. Broadly speaking, the problem with “complexity science” is that it is trying to study “complex systems”. But the only meaningful definition of “complex system” is a system that is not currently amenable to mathematical analysis. (Note this not always the definition that “complexity scientists” seem to actually use, since they like to talk about things like the Ising model which are not only well understood and long studied by physicists, but was actually exactly solved in 1944!) Trying to study the set of all “complex systems” is a bit like trying to study the set of animals that aren’t jellyfish, snails, lemurs or sting rays.

The concepts developed by “complexity scientists” are usually either well-known and understood concepts from physics and mathematics (such as “phase transition”, “non-linear”, “non-equilibrium”, “non-ergodicity”, “criticality”, “self-similarity”) or else so hopelessly vague that as to be useless (“complexity”,”emergence”,”non-reducibility”,”self-organization”). If you want to learn about the former topics I would just recommend reading actual textbooks written by and aimed at physicists and mathematicians. For instance, I particularly like Nonlinear Dynamics and Chaos by Strogatz, if you want to understand dynamical systems, and David Tong’s lecture notes on Statistical Physics and Statistical Field Theory if you want to understand phase-transitions and critical phenomena.

Note that none of these concepts are new. Even the idea of applying these concepts to the social sciences is hardly novel, see this review for example. Note the lack of hype, and lack of buzz words.

Unfortunately, the research that I’ve seen under the moniker of “complexity science” uses these (precise, limited in scope) concepts both liberally and in a facile way. As a single example, let’s have a look at “scaling laws”. Scaling laws are symptoms of critical behavior, and, as already mentioned such critical phenomenon has long been studied by physicists. If you look at empirical datasets (such as those of city sizes, or how various biological features scale with the size of an animal) sometimes you also find power-laws, and so naturally we might try to claim that these are also “critical systems”. But this plausible idea doesn’t seem to work in reality, for both for theoretical and empirical reasons.

The theoretical problem is that, pretty much all critical systems in physics require fine-tuning. For instance, you might have to dial the temperature and pressure of your gas to really specific values in order to see the behavior. There have been attempts to find models where we don’t need to fine-tune, and this is known as “self-organized criticality”, but these have basically all failed. Models which are often claimed to posses “self-organized criticality”, such as the forest-fire model, do not actually have this behavior. On the empirical side, most purported “power-laws” are, in practice, not obviously power-laws. A long discussion of this can be found here but essentially the difficult is that it is hard in practice to distinguish power-laws from other plausible distributions, such as log-normals.

If we want to talk about the hopelessly vague topics, well, there is really nothing much to be said about them, either by complexity scientists or by anyone else. To pick on “emergence”, for the moment, I think this post from The Sequences sums up nicely the emptiness of this word. There is notion of “emergence” that does appear in physics, known as “effective field theory”, which is very central to our current understanding of both particle and condensed matter physics. You can find this discussed in any quantum field theory textbook (I particularly like Peskin & Schroeder). For some reason I’ve never seen complexity scientists discuss it, which is strange, since this is the precise mathematical language physicists use to describe the emergence of large-scale behavior in physical systems.

TLDR There is no secret sauce to studying complicated systems, and “complexity science” has not made any progress on this front. To paraphrase a famous quote, “The part that is good is not original, and the part that is correct is not original (and also misapplied).”

• I don’t think so. The “immeasurability” of the future that Vaden has highlighted has nothing to do with the literal finiteness of the timeline of the universe. It has to do, rather, with the set of all possible futures (which is provably infinite). This set is immeasurable in the mathematical sense of lacking sufficient structure to be operated upon with a well-defined probability measure. Let me turn the question around on you: Suppose we knew that the time-horizon of the universe was finite, can you write out the sample space, $\sigma$-algebra, and measure which allows us to compute over possible futures?

It certainly not obvious that the universe is infinite in the sense that you suggest. Certainly nothing is “provably infinite” with our current knowledge. Furthermore, although we may not be certain about the properties of our own universe, we can easily imagine worlds rich enough to contain moral agents yet which remain completely finite. For instance, you could image a cellular automata with a finite grid size and which only lasted for a finite duration.

However, perhaps the more important consideration is the in principle set of possible futures that we must consider when doing EV calculations, rather than the universe we actually inhabit, since even if our universe is finite we would never be able to convince our selves of this with certainty. Is it this set of possible futures that you think suffers from “immeasurability”?

• I agree with your criticism of my second argument. What I should have instead said is a bit different. There are actions whose value decreases over time. For instance, all else being equal it is better to implement a policy which reduces existential risk sooner rather than later. Patient philanthropy makes sense only if either (a) you expect the growth of your resources to outpace the value lost by failing to act now, or (b) you expect cheaper opportunities to arise in the future. I don’t think there are great reasons to believe either of these is true (or indeed false, I’m not very certain on the issue).

There are two issues with knowledge, and I probably should have separated them more clearly. The more important one is that the kind of decision-relevant information Will is asking for, that is, knowing when and how to spend your money optimally, may well just be unattainable. Optimal strategies with imperfect information probably look very different from optimal strategies with perfect information.

A secondary issue is that you actually need to generate the knowledge. I agree it is unclear whether Will is considering the knowledge problem as part of “direct” or “patient” philanthropy. But since knowledge production might eat up a large chunk of your resources, and since some types of knowledge may be best produced by trying to do direct work, plausibly the “patient philanthropist” ends up spending a lot of resources over time. This is not the image of patient philanthropy I originally had, but maybe I’ve been misunderstanding what Will was envisaging.

• I can’t speak for why other people down-voted the comment but I down-voted it because the arguments you make are overly simplistic.

The model you have of philanthropy is that on an agent in each time period has the choice to either (1) invest or (2) spend their resources, and then getting a payoff depending on how influential″ the time is. You argue that the agent should then save until they reach the most influential″ time, before spending all of their resources at this most influential time.

I think this model is misleading for a couple of reasons. First, in the real world we don’t know when the most influential time is. In this case the agent may find it optimal to spend some of their resources at each time step. For instance direct philanthropic donations may give them a better understanding in the future of how influentialness varies (ie, if you don’t invest in AI safety researchers now, how will you ever know whether/​when AI safety will be a problem?) You may also worry about “going bust”: if while you are being patient, an existential catastrophe (or value lock-in) happens, then the patient long-termist looses their entire investment.

Perhaps one way to phrase how important this knowledge problem is to finding the optimal strategy is to think about it as analogous to owning stocks in a bubble. You strategy is that we should sell at the market peak, but we can’t do that if we don’t know when that will be.

Second, there are very plausible reasons why now may be the best time to donate. If we can spend money today to permanently reduce existential risk, or to permanently improve the welfare of the global poor, then it is always more valuable to do that action ASAP rather than wait. Likewise we plausibly get more value by working on biorisk, AI safety, or climate change today then we will in 20 years.

Third, the assumption of no diminishing marginal returns is illogical. We should be thinking about how EAs as a whole should spend their money as a whole. As an individual, I would not want to hold out for the most influential time if I thought everyone else was doing the same, and of course as a community we can coordinate.

• I should also point out that, if I’ve understood your position correctly Carl, I agree with you. Given my second argument, that a prior we have something like 1 in a trillion odds of being the most influential, I don’t think we should end up concluding much about this.

Most importantly, this is because whether or not I am the most influential person is not actually relevant decision making question.

But even aside from this I have a lot more information about the world than just a prior odds. For instance, any long-termist has information about their wealth and education which would make them fairly exceptional compared to the average human that has ever lived. They also have reasonable evidence about existential risk this century and plausible (for some loose definition of plausible) ways to influence this. At the end of the day each of us still has low odds of being the most influential person ever, but perhaps with odds more in the 1 in 10 million range, rather than 1 in a trillion.

• In his first comment Will says he prefers to frame it as “influential people” rather than “influential times”. In particular if you read his article (rather than the blog post), then in the end of section 5 he says he thinks it is plausible that the most influential people may live within the next few thousand years, so I don’t his odds that this century is the most influential can be very low (at a guess, one in a thousand?). I might be wrong though; I’d be very curious to know what Will’s prior is that the most influential person will be alive this century.

• I’m confused as to what your core outside-view argument is Will. My initial understanding of it was the following:
(A1) We are in a potentially large future with many trillions of trillions of humans
(A2) Our prior should be that we are randomly chosen amongst all living humans
then we conclude that
(C) We should have extremely low a prior odds of being amongst the most influential
To be very crudely quantitative about this, multiplying the number of humans on earth by the number of stars in the visible universe and the lifetime of the Earth, we quickly end up with estimates of ~1e38 total humans, and so priors on the order of ~1e-38.

As Buck points out, this argument doesn’t work unless you are willing to also accept with similarly extremely likelihood that the fact we appear to be very early humans is wrong. Otherwise the sheer weight of 1e38 pushes you extremely strongly to the conclusion that either (A1) is false or that we are almost certainly in a simulation.

Perhaps a somewhat different argument is closer to what you actually think. Here I’ve tried to frame the argument in a way that I think both you and Buck would find reasonable:
(A1′) A prior it is plausible that the most influential human is early. For simplicity, let’s say we have a 10% prior that the most influential human lives while the majority of humanity is still all on earth.
(A2′) The number of humans that will be alive up to the end of this period is plausible on the scale of a 100 billion people.
(A3′) Our evidence that humanity is restricted to one planet is incontrivertible (ie, no simulation)
We now conclude that
(C’) We should have low, but not absurdly astronomically low odds of being amongst the most influential humans

Note that compared to the previous argument, the a prior odds on being the most influential person is now 1e-10, so our earliness essentially increases our belief that we are the most influential by something like 1e28. But of course a 1-in-a-100 billion prior is still pretty low, and you don’t think our evidence is sufficiently strong to signficantly reduce it.

Do you agree with this argument Will? Or have a I misunderstood you?

• I’m curious what numbers you are using for Europe’s growth between 1000-1700; I didn’t think European growth over that period was particularly unusual. It is worth remembering that Europe in 1000 (particularly northern Europe) was a backwater and so benefitted from catchup growth relative to (say) China. I also don’t know how much of European growth was driven by extensive growth in eastern Europe, which doesn’t seem to be relevant that to the great divergence.

Arguments against the idea that Europe c1700 was technologically ahead of the rest of Eurasia (r at least, China) are common in the great divergence literature. A good recent discussion is Chapter 16 of A Culture of Growth by Mokyr; he discusses various similarities and differences between the two regions around 1700. For detailed discussion focussed on military questions, see The Gunpowder Age by Andrade and Why did Europe conquer the world? by Hoffman, both of which argue that the gap between European and Chinese military technology was not very large during the 1600s.

For what it’s worth I think Europe development was distinct from previous economic efflorescences in so far as it took place in the context of a fractured political landscape. Most other examples (Rome, Abbasid caliphate, many Chinese dynasties) seem to be driven by political unification allowing the growth and diversification of markets; a discussion focussed on the roman example can be found in The Roman Market Economy by Temin. This seems different to the situation in Europe c1700.

For what it’s worth it seems to me that the most plausible explanations for the great divergence are rooted in European fragmentation. This allowed a number of different economic, political, and cultural arrangements to be explored while competitive pressure encouraged more efficient institutions to be adopted. A recent discussion of this can be found in Escape from Rome by Schiedel, but the argument is made in many other places and underpins a number of other more proximate explanations of the great divergence (including Mokyr’s cultural/​institutional explanation).

• There seems to be a major disconnect between the Hyperbolic Growth Hypothesis and the great divergence literature. If we take the Hyperbolic Growth Hypothesis seriously, it seems that there is really little to explain about the industrial revolution. It is just an inevitable consequence of hyperbolic growth and is not qualitatively distinct from what occured before. Although I’m not an economic historian, I have read a number of books on the great divergence and none of them seem to agree with that analysis. They may be disagreement about the causes and the precise timeline, but not about the existence of a question to be answered.

As to why they believe this, I think it essentially boils down to the fact that if we look at the historical record, it seems that the industrial revolution occuring c1800 was highly contingent. It seems unlikely that an observer in 500AD, even with excellent data about the past and detailed knowledge of future possible technologies, could have simply extrapolated growth trends to predict the industrial revolution. We know of a number of economically advanced societies which didn’t industrialize, and indeed didn’t appear to be on the path to industrialization. Examples include early modern China, Japan, or the Ottoman empire, or more tenously Song China or Early Imperial Rome. If northwestern Europe was more like China in the year 1600, industrialization may have taken much longer, even though in that conterfactual universe economic growth up to that point may have been similar to our own universe. Ditto for a conterfactual where the Americas didn’t exist. So it seems that the Hyperbolic Growth Hypothesis proves too much, and isn’t compatible with what we actually know about the industrial revolution.