Why is horn weaker than diamond?
EliezerYudkowsky
Karma: 3,631
I agree that “biology sticks to ionic bonds and static cling” was badly put because lignin, and I’ll retire that one.
I’m not sure what’s a truer analogy than static cling for hydrophobia as a force holding things together which the general audience has any experience with. Macroscopic experience of hydrophobia is, like, oil collecting on the surface of water, which isn’t experienced as a binding force the way that static cling is.
I’m sort of skeptical that you could write something that works as science communication for a general audience, though lord knows I’m not necessarily succeeding either. The key valid ideas to be communicated are:
There exists a level above biology for molecular systems, greatly superior in terms of strength and energy density. This sets a lower bound on how a very smart and uncaring entity could kill you, which looks like it attacking you with micron-diameter robots, which looks like everyone on Earth falling over dead in the same second.
The designed micron-diameter thingies can easily kill you, where bacteria can’t, because the designed thingies can more easily rip apart human cell membranes or white blood cells made of flimsier materials. They can do that because human cell membranes are held together by static cling, as are bacterial cells; whereas the ideal limits of what micron-sized engines can be put together are more like “diamond”.
This design space isn’t accessible to natural selection despite being physically possible, because evolutionary biology has an incredibly hard time designing systems like freely rotating wheels; for reasons that generalize to evolution not creating airborne cell-engines with solid covalently bonded shells and manipulator ports. My attempt to compress “Why?” down to something maybe overly pithy is “Because shallow energy gradients are more densely connected in the design space of simple mutations than deep energy gradients.”
Now, instead of talking about human cell membranes being held together by static cling, I could talk about extremely thin metallic twisty-tie wires with some magnetized sections that help them fold up together into particular configurations in a barrel of magnetized ball bearings. Your suggestion above for science communication is that this is a great thing to mention, because it helps convey the following interesting truth: if we churn the ball bearings hard enough to unfold the twisty tie, it’ll sometimes fold right back up into the same shape again once we stop churning!
This more complicated metaphor may legit add something to an explanation of organic chemistry. I don’t disagree that it’s cool, or important to organic chemistry proper.
From the perspective of explaining how you die when you confront an uncaring mind that thinks smarter and much faster than humanity, it doesn’t add anything not already contained in “cell membranes are held together by static cling”.
Why is flesh weaker than diamond? Diamond is made of carbon-carbon bonds. Proteins also have some carbon-carbon bonds! So why should a diamond blade be able to cut skin?
I reply: Because the strength of the material is determined by its weakest link, not its strongest link. A structure of steel beams held together at the vertices by Scotch tape (and lacking other clever arrangements of mechanical advantage) has the strength of Scotch tape rather than the strength of steel.
Or: Even when the load-bearing forces holding large molecular systems together are locally covalent bonds, as in lignin (what makes wood strong), if you’ve got larger molecules only held together by covalent bonds at interspersed points along their edges, that’s like having 10cm-diameter steel beams held together by 1cm welds. Again, barring other clever arrangements of mechanical advantage, that structure has the strength of 1cm of steel rather than 10cm of steel.
Bone is stronger than wood; it runs on a relatively stronger structure of ionic bonds, which are no locally weaker than carbon bonds in terms of attojoules of potential energy per bond. Bone is weaker than diamond, then, because… why?
Well, partially, IIUC, because calcium atoms are heavier than carbon atoms. So even if per-bond the ionic forces are strong, some of that is lost in the price you pay for including heavier atoms whose nuclei have more protons that are able to exert the stronger electrical forces making up that stronger bond.
But mainly, bone is so much weaker than diamond (on my understanding) because the carbon bonds in diamond have a regular crystal structure that locks the carbon atoms into relative angles, and in a solid diamond this crystal structure is tesselated globally. Hydroxyapatite (the crystal part of bone) also tesselates in an energetically favorable configuration; but (I could be wrong about this) it doesn’t have the same local resistance to local deformation; and also, the actual hydroxyapatite crystal is assembled by other tissues that layer the ionic components into place, which means that a larger structure of bone is full of fault lines. Bone cleaves along the weaker fault line, not at its strongest point.
But then, why don’t diamond bones exist already? Not just for the added strength; why make the organism look for calcium and phosphorus instead of just carbon?
The search process of evolutionary biology is not the search of engineering; natural selection can only access designs via pathways of incremental mutations that are locally advantageous, not intelligently designed simultaneous changes that compensate for each other. There were, last time I checked, only three known cases where evolutionary biology invented the freely rotating wheel. Two of those known cases are ATP synthase and the bacterial flagellum, which demonstrates that freely rotating wheels are in fact incredibly useful in biology, and are conserved when biology stumbles across them after a few hundred million years of search. But there’s no use for a freely rotating wheel without a bearing and there’s no use for a bearing without a freely rotating wheel, and a simultaneous dependency like that is a huge obstacle to biology, even though it’s a hardly noticeable obstacle to intelligent engineering.
The entire human body, faced with a strong impact like being gored by a rhinocerous horn, will fail at its weakest point, not its strongest point. How much evolutionary advantage is there to stronger bone, if what fails first is torn muscle? How much advantage is there to an impact-resistant kidney, if most fights that destroy a kidney will kill you anyways? Evolution is not the sort of optimizer that says, “Okay, let’s design an entire stronger body.” (Analogously, the collection of faults that add up to “old age” is large enough that a little more age resistance in one place is not much of an advantage if other aging systems or outward accidents will soon kill you anyways.)
I don’t even think we have much of a reason to believe that it’d be physically (rather than informationally) difficult to have a set of enzymes that synthesize diamond. It could just require 3 things to go right simultaneously, and so be much much harder to stumble across than tossing more hydroxyapatite to lock into place in a bone crystal. And then even if somehow evolution hit on the right set of 3 simultaneous mutations, sometime over the history of Earth, the resulting little isolated chunk of diamond probably would not be somewhere in the phenotype that had previously constituted the weakest point in a mechanical system that frequently failed. If evolution has huge difficulty inventing wheels, why expect that it could build diamond chainmail, even assuming that diamond chainmail is physically possible and could be useful to an organism that had it?
Talking to the general public is hard. The first concept I’m trying to convey to them is that there’s an underlying physical, mechanical reason that flesh is weaker than diamond; and that this reason isn’t that things animated by vitalic spirit, elan vital, can self-heal and self-reproduce at the cost of being weaker than the cold steel making up lifeless machines, as is the price of magic imposed by the universe to maintain game balance. This is a very natural way for humans to think; and the thing I am trying to come in and do is say, “Actually, no, it’s not a mystical balance, it’s that diamond is held together by bonds that are hundreds of kJ/mol; and the mechanical strength of proteins is determined by forces a hundred times as weak as that, the part where proteins fold up like spaghetti held together by static cling.”
There is then a deeper story that’s even harder to explain, about why evolution doesn’t build freely rotating wheels or diamond chainmail; why evolutionary design doesn’t find the physically possible stronger systems. But first you need to give people a mechanical intuition for why, in a very rough intuitive sense, it is physically possible to have stuff that moves and lives and self-repairs but is strong like diamond instead of flesh, without this violating a mystical balance where the price of vitalic animation is lower material strength.
And that mechanical intuition is: Deep down is a bunch of stuff that, if you could see videos of it, would look more like tiny machines than like magic, though they would not look like familiar machines (very few freely rotating wheels). Then why aren’t these machines strong like human machines of steel are strong? Because iron atoms are stronger than carbon atoms? Actually no, diamond is made of carbon and that’s still quite strong. The reason is that these tiny systems of machinery are held together (at the weakest joints, not the strongest joints!) by static cling.
And then the deeper question: Why does evolution build that way? And the deeper answer: Because everything evolution builds is arrived at as an error, a mutation, from something else that it builds. Very tight bonds fold up along very deterministic pathways. So (in the average case, not every case) the neighborhood of functionally similar designs is densely connected along shallow energy gradients and sparsely connected along deep energy gradients. Intelligence can leap long distances through that design space using coordinated changes, but evolutionary exploration usually cannot.
And I do try to explain that too. But it is legitimately more abstract and harder to understand. So I lead with the idea that proteins are held together by static cling. This is, I think, validly the first fact you lead with if the audience does not already know it, and just has no clue why anyone could possibly possibly think that there might even be machinery that does what bacterial machinery does but better. The typical audience is not starting out with the intuition that one would naively think that of course you could put together stronger molecular machinery, given the physics of stronger bonds, and then we debate whether (as I believe) the naive intuition is actually just valid and correct; they don’t understand what the naive intuition is about, and that’s the first thing to convey.
If somebody then says, “How can you be so ignorant of chemistry? Some atoms in protein are held together by covalent bonds, not by static cling! There’s even eg sulfur bonds whereby some parts of the folded-spaghetti systems end up glued together with real glue!” then this does not validly address the original point because: the underlying point about why flesh is more easily cleaved than diamond, is about the weakest points of flesh rather than the strongest points in flesh, because that’s what determines the mechanical strength of the larger system.
I think there is an important way of looking at questions like these where, at the final end, you ask yourself, “Okay, but does my argument prove that flesh is in fact as strong as diamond? Why isn’t flesh as strong as diamond, then, if I’ve refuted the original argument for why it isn’t?” and this is the question that leads you to realize that some local strong covalent bonds don’t matter to the argument if those bonds aren’t the parts that break under load.
My main moral qualm about using the Argument From Folded Spaghetti Held Together By Static Cling as an intuition pump is that the local ionic bonds in bone are legitimately as strong per-bond as the C-C bonds in diamond, and the reason that bone is weaker than diamond is (iiuc) actually more about irregularity, fault lines, and resistance to local deformation than about kJ/mol of the underlying bonds. If somebody says “Okay, fine, you’ve validly explained why flesh is weaker than diamond, but why is bone weaker than diamond?” I have to reply “Valid, iiuc that’s legit more about irregularity and fault lines and interlaced weaker superstructure and local deformation resistance of the bonds, rather than the raw potential energy deltas of the load-bearing welds.”
I broadly endorse this reply and have mostly shifted to trying to talk about “covalently bonded” bacteria, since using the term “diamondoid” (tightly covalently bonded CHON) causes people to panic about the lack of currently known mechanosynthesis pathways for tetrahedral carbon lattices.
In this case, the positions from the last bullseyes become reversed. The doomer will argue that that AI might start off incapable, but will quickly evolve into a capable super-AI, following path A. Whereas I will retort that it might get more powerful, but that doesn’t guarantee it will ever actually end up being world domination worthy.
No, the doomer says, “If that AI doesn’t destroy the world, people will build a more capable one.” Current AIs haven’t destroyed the world. So people are trying to build more capable ones.
There is some weird thing here about people trying to predict trajectories, not endpoints; they get as far as describing, in their story, an AI that doesn’t end the world as we know it, and then they stop, satisfied that they’ve refuted the doomer story. But if the world as we know it continues, somebody builds a more powerful AI.
The first object-level issue the author talks about is whether the brain is close to the Landaeur limit. No particular issue is cited, only that somebody else claimed a lot of authority and claimed I was wrong about something, what exactly is not shown.
The brain obviously cannot be operating near the Landaeur limit. Thousands of neurotransmitter molecules and thousands of ions need to be pumped back to their original places after each synaptic flash. Each of these is a thermodynamically irreversible operation and it staggers the imagination that every ion pumped en masse back out of some long axon or dendrite, after ions flooded en masse into it to propagate electrical depolarization, is part of a well-designed informational algorithm that could not be simplified. Any calculation saying that biology is operating close to the Landaeur limit has reached a face-value absurdity.
Of course, this may not seem to address anything, since OP failed to state what I was putatively wrong about and admits to not understanding it themselves; I can’t refute what isn’t shown.
The first substantive criticism OP claims to understand theirself is on Zombies.
I say:
Your “zombie”, in the philosophical usage of the term, is putatively a being that is exactly like you in every respect—identical behavior, identical speech, identical brain; every atom and quark in exactly the same position, moving according to the same causal laws of motion—except that your zombie is not conscious.
The author would have you believe this is a ludicrous straw position.
I invite anyone to simply read the opening paragraphs of the SEP encyclopedia entry on P-zombies:
If zombies are to be counterexamples to physicalism, it is not enough for them to be behaviorally and functionally like normal human beings: plenty of physicalists accept that merely behavioral or functional duplicates of ourselves might lack qualia. Zombies must be like normal human beings in all physical respects, and they must have the physical properties that physicalists suppose we have. This requires them to be subject to the causal closure of the physical, which is why their supposed lack of consciousness is a challenge to physicalism. If instead they were to be conceived of as creatures whose behavior could not be explained physically, physicalists would have no reason to bother with the idea: there is plenty of evidence that, as epiphenomenalists hold, our movements actually are explicable in physical terms (see e.g. Papineau 2002).
This is a debate that has gone on for very long in philosophy. I’d say it’s gone on too long.
But whether or not particular thought experiments, by seeming metaphysically possible, license other conclusions about metaphysics, is exactly the entire substance. The base thought experiment is not or should not be in dispute: it’s a being whose physics duplicate the physics of a human being including the causal closure of what is said to be ‘physics’, i.e., all of the causes of behavior are included into the p-zombie. Some people go on at fantastic length from this to say that it demonstrates the possibility of an extra consciousness that they call “epiphenomenal”, and some say that it demonstrates the possibility of a nonphysical consciousness that they don’t call “epiphenomenal”, but it’s my position that somewhere along the way of a long argument they have dropped the ball on the original thought experiment; whatever they call “consciousness” that isn’t in the supposed p-zombie, it can’t be among the causes of why we talk about consciousness, or why our verbally reportable stream of thought talks about consciousness, etc, because the zombie behaves outwardly like we do and also includes the minimal closure of the causes of that physical behavior.
The author of the above post has misrepresented what my zombies argument was about. It’s not that I think philosophers openly claim that p-zombies demonstrate epiphenomenalism; it’s that I think philosophers are confused about what this thought experiment demonstrates.
The author having been shown to be wrong on the first points addressed, which I chose in order rather than selectively sampling, I hope you accept this as obvious evidence that the rest would be no better if you looked into them in detail or I responded in detail. For a post claiming to show that I’m often grossly wrong, actual quotes from me, with linked context and dates attached, are remarkably thin on the ground.
You will mark that in this comment I first respond to a substantive point and show it to be mistaken before I make any general criticism of the author; which can then be supported by that previously shown, initial, first-thing, object-level point. You will find every post of the Less Wrong sequences written the same way.
As the entire post violates basic rules of epistemic conduct by opening with a series of not-yet-supported personal attacks, I will not be responding to the rest in detail. I’m sad about how anything containing such an egregious violation of basic epistemic conduct got this upvoted, and wonder about sockpuppet accounts or alternatively a downfall of EA. The relevant principle of epistemic good conduct seems to me straightforward: if you’ve got to make personal attacks (and sometimes you do), make them after presenting your object-level points that support those personal attacks. This shouldn’t be a difficult rule to follow, or follow much better than this; and violating it this hugely and explicitly is sufficiently bad news that people should’ve been wary about this post and hesitated to upvote it for that reason alone.
Seems disingenuous to write a post this long, and not mention the obvious notion that more complicated and poorly known domains (than chess) will place weaker minds at a greater disadvantage to more powerful ones (compared to chess). This is why a superintelligence could not beat a short Python program at logical Tic-Tac-Toe, and 21st-century Russia could easily roll over 11th-century Earth with tanks and if need be nuclear weapons (even though both of those populations are the same species). Chess occupies a middle ground between those two points on the spectrum, but I’d say lies most of the way toward Tic-Tac-Toe.
Pausing AI Developments Isn’t Enough. We Need to Shut it All Down
GPTs are Predictors, not Imitators
Somehow I never thought about it that way. Point conceded.
The analogy survives and if anything becomes more meaningful, but is now harder to explain to a general audience: After training humans exclusively on inclusive genetic fitness, with a correlation in the outer environment to high-calorie foods, humans ended up preferring something that didn’t exist in the ancestral environment, lacks correlations to micronutrients that were reliably in ample supply in the ancestral environment and didn’t need to be optimized over, has some resemblance to things that were important/scarce like the taste of sugar and salt and fat (if the sugar hasn’t been replaced with allulose), but where it ultimately depends on properties like “the ice cream is cold rather than melted” that don’t match to anything obvious at a surface glance about the ancestral environment; and on the whole, the thing that starts to max out human tastebuds seems almost impossible to have called in advance by any simple means.
If you want the old form of the analogy, “male humans scrolling Tumblr porn” works (2D images not present in ancestral environment, Coolidge effect superstimulated). Hopefully I or somebody can think of a more general-audiences-friendly transparent example of a superstimulus than that one.
FYI: IIRC/IIUC, Bryk is the one who made up the thing about my having a harem of submissive mathematicians whom I called my “math pets”. This is false; people sufficiently connected within the community will know that it is false, not least since it’d be widely known and I wouldn’t have denied it if it were true. I am not sure what to do about it simply, if someone’s own epistemic location is such that my statements there are unknowable to them as being true.
It is known to me that Bryk has gone on repeating the “math pets” allegation, including to journalists, long after it should’ve been clear to her that it was not true.
My own understanding of proper procedure subsequent to this would be to treat Bryk as somebody having made a known false allegation, especially since I don’t know of any corresponding later-verified/known-true allegations that she was first to bring forth; and that this implies we ought to cross everything alleged by Bryk off any such lists, unless there’s independent witnesses for it, in which case we can consider those witnesses and also reconsider the future degree to which Bryk ought to (not) be considered as an evidential source.
(If I am recalling correctly that Jax started the “math pets” thing.)
IIRC, Jax is Bryk is the one who made up the “math pets” allegation against me, which hopefully everyone knows to be false. I don’t know anything about the state of the rest of the allegations against Michael, but if I’m recalling correctly that Jax is that particular known-false-accuser, we probably want to subtract anything from Jax and then evaluate the rest of the list.
The usual argument, which I think is entirely valid, and has been delivered by famouser and more famously reputable people if you don’t want to trust me about it, was named the “Gell-Mann Amnesia Effect” by Richard Feynman. Find something that you are really, truly an expert on. Find an article in TIME Magazine about it. Really take note of everything they get wrong. Try finding somebody who isn’t an expert and see what their takeaways from the article were—what picture of reality they derive without your own expertise to guide them in interpretation.
Then go find what you think is a pretty average blog post by an Internet expert on the same topic.
It is, alas, not something you can condense into a single webpage, because everybody has their own area of really solid expertise, even if it’s something like “the history of Star Trek TOS” because their day job doesn’t lead them into the same level of enthusiasm. Maybe somebody should put together a set of three comparisons like that, from three different fields—but then the skeptics could worry it was all cherry-picked unusual bad examples, even if it hadn’t been.
I will note that I do think that the great scientists of recent past generations have earned more of our respect than internationally famous journalistic publications, and those scientists did not speak kindly of their coverage of science—and that was before the era of clickbait, back when the likes of the New York Times kept to notably higher editorial standards.
I think you can talk to any famous respectable person in private, and ask them if there should be a great burden of skepticism about insinuating that a “major international publication” like TIME Magazine might be skewing the truth the way that Aella describes, and the famous respectable person (if they are willing to answer you at all) will tell you that you should not hold that much trust towards TIME Magazine.
I’d absolutely bring the same kind of skepticism. I would refuse to read a TIME expose of supposed abuses within LDS, because I would expect it to take way too much work to figure out what kind of remote reality would lie behind the epstemic abuses that I’d expect TIME (or the New York Times or whoever) would devise. If I thought I needed to know about it, I would poke around online until I found an essay written by somebody who sounded careful and evenhanded and didn’t use language like journalists use, and there would then be a possibility that I was reading something with a near enough relation to reality that I could end up closer to the truth after having tried to do my own mental corrections.
I want to be very clear that this is not my condescending advice to Other People who I think are stupider than I am. I think that I am not able to read coverage in the New York Times and successfully update in a more truthward direction, after compensating for what I think their biasing procedures are. I think I just can’t figure out the truth from that. I don’t think I’m that smart. I avoid clicking through, and if it’s an important matter I try to find a writeup elsewhere instead.
I’ve had worse experiences with coverage from professional journalists than I have from random bloggers. My standard reply to a journalist who contacts me by email is “If you have something you actually want to know or understand, I will answer off-the-record; I am not providing any on-the-record quotes after past bad experiences.” Few ever follow up with actual questions.
A sincere-seeming online person with a blog can, any time they choose to, quote you accurately and in context, talk about the nuance, and just generally be truthful. Professional journalists exist in a much stranger context that would require much longer than this comment to describe.
I mean the human tendency, not the EA tendency. TIME does it because it’s effective on their usual audience. EAs, evidently, have not risen above that.
If you think there’s an actual problem, I think the correct avenue is doing a real investigation and a real writeup. Trying to “steelman” a media version of it, that is going to be incredibly and deliberately warped, adversarially targeted at exploiting the audience’s underestimate of its warping by experienced adversaries, strikes me as a very wrong move. And it’s just legit hard to convey how very wrong of a move it is, if you’ve never been the subject of that kind of media misrepresentation in your personal direct experience, because you really do underestimate how bad it is until then. Aella did. I did.
I also attest that Aella is, if anything, severely underconveying the extent to which this central thesis is true. It’s really really hard to convey until you’ve lived that experience yourself. I also don’t know how to convey this to people who haven’t lived through it. My experience was also of having been warned about it, but not having integrated the warnings or really actually understood how bad the misrepresentation actually was in practice, until I lived through it.
What I’m pointing at there is that for strength/weight purposes, using big calcium nuclei to create stronger individual bonds in bone, is like making a steel beam stronger by putting more steel into it; the strength costs weight.