Overall, this seems like a weak criticism worded strongly. It looks like the opposition here is more to the moniker of Complexity Science and its false claims of novelty but not actually to the study of the phenomenon that fall within the Complexity Science umbrella. This is analogous to a critique of Machine Learning that reads “ML is just a rebranding of Statistics”. Although I agree that it is not novel and there is quite a bit of vagueness in the field, I disagree on the point that Complexity Science has not made progress.
I think the biggest utility of Complexity Science comes in breaking disciplinary silos. Rebranding things to Complexity Science, just brings all the ideas on systems from different disciplines together under one roof. If you are a student, you can learn all these phenomena in one course or degree. If you are a professor, you can work on anything that relates to Complex Systems phenomena if you are in a Complexity department. The flip side of it is, you might end up living in a world of hammers without nails—you would just have a bunch of tools without a strong domain knowledge in any of the systems that you are studying.
My take on Complexity Science is that it is a set of tools to be used in the right context. For your specific context, some or none of the tools of Complexity Science can be useful. Where Complexity Science falls apart for me is when it tries to lose all context and generalize to all systems. I think the OP here is trying to stay within context. The post is just saying we can build ABMs to approach some specific EA cause areas. So I am more or less onboard with this post.
On a final note, I am in agreement with your critique on abuse of Power Laws. There are too many people that just make a log-log plot, look at the line and exclaim “Power law!”. The Clauset-Shalizi-Newman paper you linked to is the citation classic here. For those who do network theory, instead of trying to prove your degree distribution is a power law, I would recommend doing Graphlet Analysis.
If the OP wants to discuss agent-based modeling, then I think they should discuss agent-based modeling. I don’t think there is anything to be gained by calling agent-based models “complex systems”, or that taking a complexity science viewpoint adds any value.
Likewise, if you want to study networks, why not study networks? Again, adding the word “complex” doesn’t buy you anything.
As I said in my original comment, part of complexity science is good: this is the idea we can use maths and physics to modeling other systems. But this is hardly a new insight. Economists, biophysicists, mathematical biologists, computer scientists, statisticians, and applied mathematicians have been doing this for centuries. While sometimes siloing can be a problem, for the most part ideas flow fairly freely between these disciplines and there is a lot of cross-pollination. When ideas don’t flow it is usually because they aren’t useful in the new field. (Maybe they rely on inappropriate assumptions, or are useful in the wrong regime, or answer the wrong questions, or are trivial and/or intractable in situations the new field cares about, or don’t give empirically testable results, or are already used by the new field in a slightly different way.) The “problem” of “siloing” that complexity science claims to want to solve is largely a mirage.
But of course, complexity science makes greater claims than just this. It claims to be developing some general insights into the workings of complex systems. As I’ve noted in my previous comment, these claims are at best just false and at worst completely vacuous. I think it is dangerous to support the kind of sophistry spouted by complexity scientists, for the same reason it is dangerous to support sophistry anywhere. At best it draws attention away from scientists who are making progress on real problems, and at worst it leads to piles of misleading and overblown hype.
My criticism is not analogous to the claim that “ML is just a rebranding of statistics”. After all, ML largely studies different topics and different questions to statistics. No, it would be as if we lived in a world without computers, and ML consisted of people waxing lyrically about how “computation” would solve learning, but then when asked how would just say basic (and sometimes incorrect) things about statistics.
@djbinder Thanks for taking the time to write these comments. No need to worry about being negative, this is exactly the sort of healthy debate that I want to see around this subject.
I think you make a lot of fair points, and it’s great to have these insights from someone with a background in theoretical physics, however I would still disagree slightly on some of them, I will try to explain myself below.
I don’t think the only meaningful definition of complex systems is that they aren’t amenable to mathematical analysis, that is perhaps a feature of them, but not always true. I would say the main hallmark is that there is a surprising level of sophisticated behaviour arising from only apparently simple rules at the level of the individual components that make up that system, and that it can be a challenge to manage and predict such systems.
It is true that the terms “complexity” and “emergence” are not formally defined, and this maybe means that they end up getting used in an overly broad way. The area of complexity science has also been a bit prone to hype. I myself have felt uncomfortable with the term “emergence” at times, it is maybe still a bit vague for my tastes, however I have landed on the opinion that it is a good way to recognise certain properties of a system and categorise different systems. I agree with Eliezer Yudkowky’s point that it isn’t a sufficient explanation of behaviour, but it is still a relevant aspect of a system to look for, and can shape expectations. The aspiration of complexity science is to provide more formal definitions of these terms. So I do agree that there is more work to do to further refine these terms. However just because these terms can’t be formally or mathematically defined yet, doesn’t mean they have no place in science. This is also true of words like “meaning” and “consciousness”, however these are still important concepts.
I think the main point of disagreement is whether “complexity science” is a useful umbrella term. I agree that plenty of valuable interdisciplinary work applying ideas from physics to social sciences is done without reference to “complexity” or “complex systems”, however by highlighting common themes between these different areas I think complexity science has promoted a lot more interdisciplinary work than would have been done otherwise. With the review paper you linked, I would be surprised if many of the authors of those papers didn’t have some connection to complexity science or SFI at some point. In fact one of the authors is director of a lab called “Center for complex networks and systems research”. Even Steven Strogatz, whose textbook you mentioned, was an external SFI professor for a while! Although it’s true that just because he’s affiliated with them doesn’t mean that complexity science can take credit for all his prior work. Most complexity scientists do not typically mention complexity or emergence much in their published papers, they will just look like rigorous papers in a specific domain. Although the flip side of this is maybe that casts into doubt the utility of these terms, as you argued. But I would say that this framing of the problem (as “complex systems” in different domains having underlying features in common) has helped to motivate and initiate a lot of this work. The area of complexity economics is a great example of this, economics has always borrowed ideas from physics (all the way back to Walrasian equilibrium), however this process had stalled somewhat in the latter half of the 20th century. Complexity science has injected a lot of new and valuable ideas into economics, and I would say this comes from the idea of framing the economy as a complex system, not just because SFI got the right people in the same room together (although that is a necessary part).
Perhaps I am just less optimistic than you about how easy it is to do good interdisciplinary work, and how much of this would happen organically in this area without a dedicated movement towards this. I maintain that complexity science is a good way to encourage researchers to push into problem areas that are less amenable to reductionism or mathematical analysis, since this is often very difficult and risky.
Anyway the main reason I wanted to write this blog post is not so that EA people go around waxing lyrical with words like “complexity” and “emergence” all the time, but to point to complexity science as an example of a successful interdisciplinary movement, which maybe EA can learn from (even just from a public relations point of view), and also to look at some of the tools from complexity science (eg. ABMs) and suggest that these might be useful. @Venkatesh makes a good point that my main recommendation here is that ABMs may be useful to apply to EA cause areas, so perhaps I should have separated that bit out into a separate forum post.
Thanks for the reply Rory! I think at this point it is fairly clear where we agree (quantitative methods and ideas from maths and physics can be helpful in other disciplines) and where we disagree (whether complexity science has new insights to offer, and whether there is a need for an interdisciplinary field doing this work separate from the ones that already exist), and don’t have any more to offer here past my previous comments. And I appreciate your candidness noting that most complexity scientists don’t mention complexity or emergence much in their published research; as is probably clear I think this suggests that, despite their rhetoric, they haven’t managed to make these concepts useful.
I do not think the SFI, at least judging from their website, and from the book Scale which I read a few years ago, is a good model of public relations that EAs should try to emulate. They make grand claims about what they have achieved which seems to me to be out of proportion to their actual accomplishments. I’m curious to hear what you think the great success stories of SFI are. The one I know the most about, the scaling laws, I’m pretty skeptical of for the reasons outlined previously. I had a look at their “Evolution of Human Languages” program, and it seems to be fringe research by the standards of mainstream comparative linguistics. But there could well be success stories that I am unfamiliar with, particularly in economics.
Overall, this seems like a weak criticism worded strongly. It looks like the opposition here is more to the moniker of Complexity Science and its false claims of novelty but not actually to the study of the phenomenon that fall within the Complexity Science umbrella. This is analogous to a critique of Machine Learning that reads “ML is just a rebranding of Statistics”. Although I agree that it is not novel and there is quite a bit of vagueness in the field, I disagree on the point that Complexity Science has not made progress.
I think the biggest utility of Complexity Science comes in breaking disciplinary silos. Rebranding things to Complexity Science, just brings all the ideas on systems from different disciplines together under one roof. If you are a student, you can learn all these phenomena in one course or degree. If you are a professor, you can work on anything that relates to Complex Systems phenomena if you are in a Complexity department. The flip side of it is, you might end up living in a world of hammers without nails—you would just have a bunch of tools without a strong domain knowledge in any of the systems that you are studying.
My take on Complexity Science is that it is a set of tools to be used in the right context. For your specific context, some or none of the tools of Complexity Science can be useful. Where Complexity Science falls apart for me is when it tries to lose all context and generalize to all systems. I think the OP here is trying to stay within context. The post is just saying we can build ABMs to approach some specific EA cause areas. So I am more or less onboard with this post.
On a final note, I am in agreement with your critique on abuse of Power Laws. There are too many people that just make a log-log plot, look at the line and exclaim “Power law!”. The Clauset-Shalizi-Newman paper you linked to is the citation classic here. For those who do network theory, instead of trying to prove your degree distribution is a power law, I would recommend doing Graphlet Analysis.
If the OP wants to discuss agent-based modeling, then I think they should discuss agent-based modeling. I don’t think there is anything to be gained by calling agent-based models “complex systems”, or that taking a complexity science viewpoint adds any value.
Likewise, if you want to study networks, why not study networks? Again, adding the word “complex” doesn’t buy you anything.
As I said in my original comment, part of complexity science is good: this is the idea we can use maths and physics to modeling other systems. But this is hardly a new insight. Economists, biophysicists, mathematical biologists, computer scientists, statisticians, and applied mathematicians have been doing this for centuries. While sometimes siloing can be a problem, for the most part ideas flow fairly freely between these disciplines and there is a lot of cross-pollination. When ideas don’t flow it is usually because they aren’t useful in the new field. (Maybe they rely on inappropriate assumptions, or are useful in the wrong regime, or answer the wrong questions, or are trivial and/or intractable in situations the new field cares about, or don’t give empirically testable results, or are already used by the new field in a slightly different way.) The “problem” of “siloing” that complexity science claims to want to solve is largely a mirage.
But of course, complexity science makes greater claims than just this. It claims to be developing some general insights into the workings of complex systems. As I’ve noted in my previous comment, these claims are at best just false and at worst completely vacuous. I think it is dangerous to support the kind of sophistry spouted by complexity scientists, for the same reason it is dangerous to support sophistry anywhere. At best it draws attention away from scientists who are making progress on real problems, and at worst it leads to piles of misleading and overblown hype.
My criticism is not analogous to the claim that “ML is just a rebranding of statistics”. After all, ML largely studies different topics and different questions to statistics. No, it would be as if we lived in a world without computers, and ML consisted of people waxing lyrically about how “computation” would solve learning, but then when asked how would just say basic (and sometimes incorrect) things about statistics.
@djbinder Thanks for taking the time to write these comments. No need to worry about being negative, this is exactly the sort of healthy debate that I want to see around this subject.
I think you make a lot of fair points, and it’s great to have these insights from someone with a background in theoretical physics, however I would still disagree slightly on some of them, I will try to explain myself below.
I don’t think the only meaningful definition of complex systems is that they aren’t amenable to mathematical analysis, that is perhaps a feature of them, but not always true. I would say the main hallmark is that there is a surprising level of sophisticated behaviour arising from only apparently simple rules at the level of the individual components that make up that system, and that it can be a challenge to manage and predict such systems.
It is true that the terms “complexity” and “emergence” are not formally defined, and this maybe means that they end up getting used in an overly broad way. The area of complexity science has also been a bit prone to hype. I myself have felt uncomfortable with the term “emergence” at times, it is maybe still a bit vague for my tastes, however I have landed on the opinion that it is a good way to recognise certain properties of a system and categorise different systems. I agree with Eliezer Yudkowky’s point that it isn’t a sufficient explanation of behaviour, but it is still a relevant aspect of a system to look for, and can shape expectations. The aspiration of complexity science is to provide more formal definitions of these terms. So I do agree that there is more work to do to further refine these terms. However just because these terms can’t be formally or mathematically defined yet, doesn’t mean they have no place in science. This is also true of words like “meaning” and “consciousness”, however these are still important concepts.
I think the main point of disagreement is whether “complexity science” is a useful umbrella term. I agree that plenty of valuable interdisciplinary work applying ideas from physics to social sciences is done without reference to “complexity” or “complex systems”, however by highlighting common themes between these different areas I think complexity science has promoted a lot more interdisciplinary work than would have been done otherwise. With the review paper you linked, I would be surprised if many of the authors of those papers didn’t have some connection to complexity science or SFI at some point. In fact one of the authors is director of a lab called “Center for complex networks and systems research”. Even Steven Strogatz, whose textbook you mentioned, was an external SFI professor for a while! Although it’s true that just because he’s affiliated with them doesn’t mean that complexity science can take credit for all his prior work. Most complexity scientists do not typically mention complexity or emergence much in their published papers, they will just look like rigorous papers in a specific domain. Although the flip side of this is maybe that casts into doubt the utility of these terms, as you argued. But I would say that this framing of the problem (as “complex systems” in different domains having underlying features in common) has helped to motivate and initiate a lot of this work. The area of complexity economics is a great example of this, economics has always borrowed ideas from physics (all the way back to Walrasian equilibrium), however this process had stalled somewhat in the latter half of the 20th century. Complexity science has injected a lot of new and valuable ideas into economics, and I would say this comes from the idea of framing the economy as a complex system, not just because SFI got the right people in the same room together (although that is a necessary part).
Perhaps I am just less optimistic than you about how easy it is to do good interdisciplinary work, and how much of this would happen organically in this area without a dedicated movement towards this. I maintain that complexity science is a good way to encourage researchers to push into problem areas that are less amenable to reductionism or mathematical analysis, since this is often very difficult and risky.
Anyway the main reason I wanted to write this blog post is not so that EA people go around waxing lyrical with words like “complexity” and “emergence” all the time, but to point to complexity science as an example of a successful interdisciplinary movement, which maybe EA can learn from (even just from a public relations point of view), and also to look at some of the tools from complexity science (eg. ABMs) and suggest that these might be useful. @Venkatesh makes a good point that my main recommendation here is that ABMs may be useful to apply to EA cause areas, so perhaps I should have separated that bit out into a separate forum post.
Thanks for the reply Rory! I think at this point it is fairly clear where we agree (quantitative methods and ideas from maths and physics can be helpful in other disciplines) and where we disagree (whether complexity science has new insights to offer, and whether there is a need for an interdisciplinary field doing this work separate from the ones that already exist), and don’t have any more to offer here past my previous comments. And I appreciate your candidness noting that most complexity scientists don’t mention complexity or emergence much in their published research; as is probably clear I think this suggests that, despite their rhetoric, they haven’t managed to make these concepts useful.
I do not think the SFI, at least judging from their website, and from the book Scale which I read a few years ago, is a good model of public relations that EAs should try to emulate. They make grand claims about what they have achieved which seems to me to be out of proportion to their actual accomplishments. I’m curious to hear what you think the great success stories of SFI are. The one I know the most about, the scaling laws, I’m pretty skeptical of for the reasons outlined previously. I had a look at their “Evolution of Human Languages” program, and it seems to be fringe research by the standards of mainstream comparative linguistics. But there could well be success stories that I am unfamiliar with, particularly in economics.