At what point can large language models start to do distillation, especially of the early LW sequences?
InquilineKea
Karma: 84
Also hi Akash!!! Nice to see you here!! :) Jeffrey Yun pointed me here! :)
more on mosaicism—https://twitter.com/jpsenescence/status/1084560766735450113
The large number of mutations with age recent studies are finding in some human tissues showcase how difficult it will be to significantly intervene in aging because we can’t easily get rid of mutant cells and replace them by pristine cells.
https://www.nature.com/articles/d41586-018-07737-8 is very deep too—actually it hints that many older cells are dominated by pro-growth/pro-survival mutations that don’t complete all the necessary conditions for cancer (but it just shows how cancer is the adaptive response of A LOT of other responses that are pro-growth/survival in ordinary cells that USUALLY don’t result in cancer...)
It’s not just tau/junk that contributes to cytoskeleton damage—the cytoskeleton is made of proteins that are easily oxidizeable in the same way that nuclear pore complexes are, and damage to NPCs don’t have tau as their primary culprit.
On moral progress—I think it’s highly plausible that future generations will not be okay with people dying due to natural causes in the same way that they’re not okay with people dying from cancer or infectious diseases.
Too much tau junk → too much cytoskeleton damage
That’s not the only thing that causes cytoskeleton damage.
Ultimately one path forward is: how do you create the data-set/papers that can be used by a new version of GPT-3 to suggest potential interventions for aging. That’s why ALL of the creative new technologies people use to treat genetic diseases or cancer (along with nanotechnology—yes UPenn people are already creating nanobots) can help, even if not originally designed for aging.
Cytoskeleton damage can be upstream/causal if it affects lysosomal positioning (just as anything that affects autophagy reaching the sites it needs to reach can be upstream/causal). It also affects cellular stiffness, which then affects whether molecules reach the places they should be reaching.
Lipofuscin can also be a secondary kind of damage too, and it doesn’t seem to adversely affect the cell too much until its concentration reaches a critical level.
Much of SENS was developed before the massive bioscience advances in understanding over the last 15 years—we can do better to adopt to what these new bioscience advances may imply, and there is a strong possibility that it’s much more complicated than you think it is and that damage to every single critical of the cell is somehow causally involved. I know scientists who criticize SENS on account of it underestimating the sheer complexity of the cell [and its attitude of not needing to know everything to fix damage] - while it is probably true that you don’t need to know everything to fix damage (especially if you look into low-hanging fruit like developmental biology/regeneration/stem cells/replacement organs), what SENS does right now is not sufficient
Abrupt cellular phase changes (see https://shiftbioscience.com/ and also Tony Wyss-Corey) that happen through life may be more impt than previously thought. I don’t doubt that more investment in SENS would have a high chance of producing something desireable, but there’s a high chance that the most consequential interventions may come through other routes.
This would only matter a lot if you want to disentangle what metabolism is doing (which is vast) and try to get it to do the impossible: prevent every single lipid and protein from going bad. I doubt even an AI god could make that happen, nevermind mere mortals equipped with what amount to fancy expert systems.
Preventing every single lipid and protein from going bad is precisely a problem that “AI” could help solve—one could envision artificially designed enzymes that can get into the cell and specifically modify every unnecessary oxidative modification.
Better funding is better than better tools. If SENS got $100 million per year starting in 2004 or even as late as 2010, we’d already have immortality in the bag or know that SENS couldn’t deliver the goods and moved on to something else.
This is a bold claim that presumes that you and others know “all the right things to do” (rather than are adaptive) + underestimate the pure complexity of biology and very few people would believe you/SENS, and the tendency of SENS foundation people to make such claims are a reason why many doubt its credibility (some of the doubt is clearly unjustified, and stems from the uncharitable motivations of skeptics, but SENS people could at least be better at qualifying their claims).
Yes, damage to long-lived NPCs can be causative given that mislocalized nucleocytoplasmic transport can be causative in reduced autophagy with age. From Autophagy in aging and longevity
immpaired nucleocytoplasmic transport and loss of nuclear integrity may derail autophagy
The proper nucleocytoplasmic transport of autophagy-inducing TFs such as TFEB by RanGTP-dependent importins and exportins and the retention of such factors in the nucleus are important processes in proper autophagic regulation. In fact, nuclear pore complexes (NPCs), which form nucleocy-toplasmic transport channels through the nuclear envelope, deteriorate with age and cause age-dependent nuclear pore leakiness in post-mitotic cells such as neurons (D’Angelo et al. 2009). The efficiency of TFEB nuclear retention may thus decrease with age, consequentially playing an important role in the age-dependent decline of autophagic activity. In the same vein, findings have highlighted the importanceRecent evidence has additionally provided further support of the importance of nucleocyto-plasmic transport in health and disease by demonstrating that pathologically-affected proteins in NDs can disrupt this process by subcellularly mislocalizing proteins and RNA (reviewed in Fahrenkrog and Harel 2018). Mislocalized proteins included NPC components and nucleocytoplasmic transporters themselves which were aberrantly partitioned to the cytoplasm, and thus inhibited from performing their functions at the nucleus by phase separated stress granules
Phase separation is important too… (an this only became a research fad 2 years ago)
Do you have evidence that this may be a cause of normal human aging rather than of progeria and aging in worms?
https://www.sciencedirect.com/science/article/abs/pii/S1566312408600528
The cytoskeleton is how the neuron is able to transport mitochondria, proteins, lysosomes, and other organelles where they’re supposed to be. Disruptions in axonal transport that happen due to cytoskeletal damage prevent the neuron from being able to transport cargo to the right places, especially to synapses). Dendritic size (and “stubs”) often shrink wrt age in part due to decreased maintenance (the smaller spines shrink/die off more).
and yes ⇒ the cytoskeleton IS how the neuron transports lysosomes to where they are needed, particularly in neurons. See https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5201012/
Although the regulation of lysosome dynamics is important in most cells, it is particularly crucial in neurons because of their extreme asymmetry and the length of axons and dendrites. Indeed, variations or mutations in components of the lysosome-positioning machinery cause various psychiatric and neurological disorders
The process of CDA involves targeting autophagosomes to lysosomes, which requires a certain kind of spatial localization that can only happen when the proper spatial cues still exist [and anything affecting autophagy is extremely central to aging reduction/”reversal”]
It’s way easier just to clear them out...
Um no, it’s much easier to fix oxidative modifications before they all irreversibly clump together into weird aggregates that become inaccessible to most enzymes. See figure at https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5536880/bin/gr1.jpg . Early intervention >> late intervention. “The reduction of lipofuscin/ceroid formation by pharmacologically decreasing oxidative stress may represent a more promising approach to the problem. ”
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5536880/
Again, better tools are nice-to-have, not must-haves.
The scope of the aging problem is so vast that we need all possible routes to discover all of the interventions (including ALL the > 200+ oxidative modifications that happen to proteins), and we may never get at all of the interventions without better tools. They might theoretically not be must-haves, but better be at the safe side and use all techniques.
From Allen Brain Institute and Janelia and other institutes, we’re seeing significant advances in our ability to image the cell and to get high throughput “-omic” data from cells, without needing too much human intervention [ever notice how Ed Boyden and Adam Marblestone are all into making better tools, even though they don’t directly do bioscience research the way other biomedical researchers do it?]. Better tools help reduce the intense labor and time costs involved in figuring out the mechanism of an intervention. They also need to be paired with better post-PDF-publication platforms as the data they generate is not easily made available via PDFs. They’re also the only way we can get to developing nanotechnology that can also play a role in identifying and removing damage.
Isn’t there a difference between creating entirely new frameworks, and just adopting frameworks to different species in parallel?
For instance, it seems that adopting frameworks to different species in parallel often ends up happening over time (as we’ve seen people gradually adopt cognitive ethology from chimpanzees and captive dolphins into wild dolphins, elephants, african grey parrots, kea [where academic labs do exist to study them], capuchin monkeys, and new Caledonian Crows). It seems that some species of animals are intensively studied, and the vast majority of others not so much (like, WHERE are the Irene Pepperbergs on storks and pelicans or Andean Condors or deer?).
One thing I’ve often noticed is how little attention is spent on studying individual orders of protists and very small animals (especially nematodes that aren’t C. elegans or most species of invertebrates), which should presumably have even more genetic diversity than what we see in charismatic megafauna.I know there is a conference specifically for tardigrades, but it hasn’t gotten much attention..
As for cognitive ethology—people like Louis Herman, Joyce Poole (of elephants), Andrea Marshall (of manta rays) all seemed to get funding through independent research institutes. It seems that sometimes they get support from sources that overlap with sources that support zoos?
John Marzluff def gets a lot of academic support for studying crows (and before that, prairie dogs)
SENS also doesn’t mention cytoskeletal aging (eg https://www.molbiolcell.org/doi/10.1091/mbc.E18-06-0362 ). It’s important because cytoskeletal proteins are among the most abundant proteins and are not easily replaceable or degradeable, given that they’re often long-lived and you can’t cut them in half without disrupting the rest of the cell [1]. You might call it a “more general version” of damage to elastin.
[1] this is also true for the most general case including structural proteins like lamin—aberrant transcripts of lamin also accumulate during aging, just not fast enough to be causative.
You might as well map out causes of aging in the most abundant proteins in https://www.proteomaps.net/index.html, with special importance placed to the extremely long-lived proteins or the ones that aren’t easily replaced or degraded.
Spliceosomes are super-relevant too given how they are upstream of everything else (William Mair has shown that dysregulation in these accelerates aging, and correcting the defects can up lifespan)
You can argue that “ER + aging”, “golgi + aging”, or any “cell process/component + aging” is going to cause some downstream effects on aging, and to fix everything, you have to “fix” the ER, fix the spliceosomes, fix the cytoskeleton, fix the golgi, fix the NPCs, fix the histones, whatever.
More than anything, the main limitation of SENS is that it doesn’t even plan for future interventions that are guided by AI/ML. Many of the smartest people I know (esp the computer scientists), for better or worse, think that a cure for aging will most likely come through AI, but they aren’t able to describe/specify how this happens—they’ll just magically think it will be. And most people in SENS don’t even plan on how to make the kinds of experimental design that will make it easier for experiments to produce vast amounts of machine-readable output that make it much easier to apply future AI/ML algorithms for ranking+testing potential therapeutics/interventions [they still only publish in journals, which produce far less data than what would be optimally useful for training “AI”]. Unless both sides have a remote idea of how make aging bioscience datasets be used to successfully “train” interventions (especially those that go beyond single molecules), this dream will never happen.
[living datasets would be nice too]
Theoretically it may be possible to evolve enzymes that can reverse most of the most common inappropriate oxidative modifications to proteins, or ones that can recognize, isolate, and clear lipofuscin deposits (though b/c they are so disorganized and hetereogeneous in size +chemical modifications, this is a difficult problem)
To start out with, funding studies to use new in-situ techniques like https://www.10xgenomics.com/spatial-transcriptomics/ can make everything in the future more machine-readable.
As for other forms of damage, it does seem that SENS focuses on repairing damage when it has already accumulated, rather than investigations into targeted interventions that can significantly slow this damage. Eg with proteasomes. The quote below is quite powerful~~
Fortunately much of the accumulated damage can be removed and the damaged proteins can be degraded and replaced by non-damaged ones. In fact, a mild degree of modification or damage to a protein makes it a better candidate for degradation by the 20S Proteasome or other proteases [33, 39, 41, 51]. However, if a protein becomes too heavily modified it becomes a very poor candidate for degradation [33, 39, 40]. Thus, while many mildly oxidized proteins are readily degradable, at least some of the age-associated (or time-associated) accumulation of damaged proteins is due to proteins which are so highly modified that they are difficult or impossible to degrade. It has been argued that it is the buildup of these non-degradable damaged proteins that causes age-related effects, however, the accumulation of oxidized proteins in cells is exponential rather than linear over time, indicating that the rise in protein oxidation is not just a product of a buildup of indigestible proteins, but a potentially reversible change in cell function [52].
The accumulation of oxidized or otherwise damaged proteins in cells during aging could be a product either of a rise in damaging conditions or a fall in the rate of removal of damaged proteins. it has been observed that over age there is a rise in mitochondrial generation of oxidants [53]. In addition, it has been shown that there is a decline in protein turnover. This decline in protein turnover is, at least in part, the product of a sharp decrease in 20S Proteasome function over age (which has been shown in a range of different tissues) [54-59]. In addition to a decline in 20S Proteasome function there is also a drop in 20S Proteasome levels over in aging which also reduces protein turnover [60-62]. The decline in 20S Proteasome function is partly the product of an increase in modification or damage to the 20S Proteasome over the course of age [54, 59-61]. For instance, it has been seen that 20S Proteasome isolated from old rats is 50% less proteolytically active than 20S Proteasome isolated from young rats [63]. As a result, not only is there a decrease in the amount of 20S Proteasome present during the aging process but there is also a decrease in the ability of the remaining 20S Proteasome to degrade the accumulating damaged proteins, thus resulting in an overall accumulation of damaged cellular proteins with age. As a result, older rats are less able to remove damaged proteins from their cells and tissues than are younger rats, which goes some way to explain the difference in the levels of protein damage found in the two animals.
Actually, the most important limiting factor is the funding of the right research. There’s just no way around that regardless of how good tools become.
Lol, everyone in the SENS program tells people”GIVE US MORE MONEY AND MAGICAL THINGS WILL HAPPEN”, but like, this seems to make other people feel like they can’t contribute to changing the mission of SENS, given that it seems to delegate all control to whoever controls SENS. I know SENS creates mission reports and such, but so far they still haven’t been great at convincing most HNWIs that SENS has made any real progress in the last 10-15 years. Funding may be necessary for progress and the chance to make a dent is probably worth it, but it’s still not convincing enough for most people.
There are far more ways to make an impact on aging than just donating more to SENS, and like, most of the anti-aging money seems to be flowing into ventures other than SENS (though expanding the number of possible routes people can take to slow aging rate is always helpful, even if it means doing silly N=1 things like injecting stem cells into one’s own body)
Is Balaji sufficiently convinced enough to donate a good fraction of his networth into SENS in the same way that Vitalik is convinced?
But we’re talking about the entire brain here, not just the part that causes PD. If 1 cubic mm of brain tissue could be replaced every day, it would take about 3,561 years to replace all of it (the brain’s volume is about 1,300,000 cubic mm).
You created a good example.. Regeneration/rejuvenation should ideally be guided by natural progenitor cells that don’t require surgical precision. I agree we should still emphasize SENS-ish issues of removing protein aggregates (both intracellular and extracellular) in the brain.
I don’t see how a “better” framework than the repair-the-damage-without-messing-with-metabolism kind can exist. If anything will work at curing aging, it has to be damage repair almost by definition; it’s the whole structure-determines-function thing I mentioned earlier.
I mentioned bowhead whales earlier, and while it may be true that they have slower metabolism, their longevities are still relevant in the same way that the longevities of birds (esp kakapo, sulfur-crested cockatoos, hyacinth macaws, and andean condors) are relevant—the birds are most relevant b/c they have faster metabolisms than humans (we know at least that they’re better at quenching mitochondrial ROS). We’ve already done A LOT to investigate the uniquely peculiar biology of naked mole rats to figure out how they are so resistant to cancer and oxidative stress (and they ARE important) [and we have papers on how their SIRT6 is different], but they ultimately age faster than humans especially b/c they still have much shorter lifespans than people and accumulate aggregated proteins at faster rates.
Many animals that have more saturated cellular membranes (high SFA to MUFA/PUFA ratio) are also more resistant to ROS and have higher longevities (though I saw a paper saying that higher levels of MUFAs are helpful)
[and with bowhead whales, it’s still important for us to know WHAT their native DNA damage and endogenous antioxidant levels are]
I don’t think we should entirely discount messing with metabolism either—hibernation induced by hydrogen sulfide might be an alternative to cryonics (aging rates do slow down during hibernation, and NASA is certainly studying induced hibernation responses in people)
Also, it turns out that membrane unsaturation doesn’t need to be targeted.
Membrane unsaturation was a weak example, I was more looking at the example of damage to cell membranes (both in their proteins and in the molecules that compose cellular membranes). Cell membranes often “stiffen” over time.
If you know of any damage that’s not covered by SENS, let me know.
Sure, what’s your email?
Protein traffic jams (https://www.sciencemag.org/news/2019/01/halt-brain-diseases-drugs-take-aim-protein-traffic-jams-kill-neurons ) isn’t mentioned by SENS and can occur even w/o protein aggregates or lipofuscin (though these def crowd out the cell and help) - it’s the same as basic protein damage to extremely long-lived proteins like nuclear pore complexes.
Bejarano E, Murray J, Wang X, Pampliega, O, Yin D, Patel B, Yuste A, Wolkoff A, Cuervo AM. Defective recruitment of motor proteins to autophagic compartments contributes to autophagic failure in aging. Aging Cell doi: 10.1111/acel.12777, 2018
What of mitochondrial transfer (https://twitter.com/Mito_News/status/1255968938648887297 + https://twitter.com/attilacsordas/status/1016775313152528386 ) as an alternative to mitoSENS? It certainly seems more feasible.
Does a model like https://twitter.com/z_chiang/status/1350933491274412032 count under SENS? I know people like George Church and David Sinclair are excited about IPSCs and epigenetic/genetic reprogramming, but having read SENS first, I’ve always been skeptical of the ability of reprogramming to clear out existing forms of cellular damage (it’s possible that it still might clear out some of the damage through inducing a more youthful transcriptome—eg one with lower inflammatory proteins and higher autophagy proteins)
SENS also has missed out on changes in glycosylation in cells (eg see
).
What about introducing deuterated PUFAs into the cell? [this may have especially high impact]
SENS also seems to concentrate its funding among a small number of labs/PIs, but what about a higher number of labs/PIs (who might be more high-risk)? Even people who do gene (or iPSC) therapy on themselves (or inject themselves with stem cells) produce valuable data that collectively have a non-zero chance of making a significant difference.
As we increase the number of tools (eg alphafold 2 advances count as another tool too), the number of possible avenues only increases (eg I know someone else who is working on trying to use RNA-based viruses to induce cells to produce the protein variants expressed by centenarians). This approach isn’t as “basic research” and may get sufficient funding on its own through self-experimenters (in the same way that people who inject stem cells into themselves are also self-experimenters, and they certainly are willing to pay a lot of $ for it)
No aging enables autocrats to stay in power indefinitely, as it is often the uncertainty of their death that leads to the failure of their regimes. Given that billions worldwide currently live under autocratic or authoritarian governments, this is a very real concern.
Among the largest nations that are most relevant to the world (or have a disproportionate ability to shape what happens to the world relative to their ability to be shaped by other countries), it only applies for China and Russia, and it’s unclear whether Xi or Putin strongly care about immortality (and even if it did, it would be unlikely to arrive quickly enough to save them). Given that the next 100 years might be the most important years ever in human history, this makes this supposition more bounded on what might happen in the next 100 years, and there aren’t that many dictators in that position. It’s also unlikely that China would become less autocratic/flexible even after Xi dies (the CCP will just have other ways to maintain its power—probably in a way similar to how North Korea barely changed after Kim Jung Il died. When an autocrat’s closest associates also die off over time, it can cause a weakening of the strong beliefs held by some of the previous generation, which might facilitate regime change.
I think this concern has a potential for strong negative downside on the tails, but it’s unclear if it is a strong negative downside in the median case (given that we know who the most relevant dictators are here, and there aren’t many). Given the increasing power disparity between China and the West, what happens in China then becomes uniquely important, so this concern may be more targeted around whether or not China’s ability to change is affected by whether or not the death of Xi’s successor (and everyone in Xi’s generation of the CCP) would significantly increase the chances of China transitioning away from the strongest downsides of autocracy or authoritarian governments (I believe China transitioning away from authoritarianism is unlikely no matter what, though the death of its autocrats over the next 100 yearsmight increase China’s chances of ultimately transitioning away from the most negative effects of authoritarian governments, such as censorship of thoughtcrimes). Conditioning everything into the far future could also time-localize (or impose an upper bound on) much of the “suffering” that comes from the mission of “transforming the identities of unreceptive people into Han Chinese” [eg people in Hong Kong now will most likely suffer in the present, but future people born in Hong Kong probably won’t “suffer” as much from not having something tangible “taken away” from them], though what China is doing now (wrt stifling dialogue) is certainly not making China’s future more robust.
It’s also possible that AI may ultimately improve social dialogue to the point that it may help the CCP get what it wants without feeling threatened if it relaxes some of its more draconian measures such as censorship. I’m not sure if prolonging the lives of China’s authoritarians is guaranteed to be a strong negatives—it certainly has issues such as being insufficiently insensitive over what it’s doing to Xinjiang/Tibet/Hong Kong (and possibly eventually Taiwan), but these issues are mostly in the now and will be unaffected by life extension in the future. What China is doing to increase its influence/power elsewhere will be done irrespective of who is in power, and it probably doesn’t have a strong desire to “take over” other countries in the way that Hitler or Stalin did (ultimately, it is more constrained by what other countries can do to it more than it is constrained by the potential deaths of its dictators, unless it had an unusually powerful/effective/ruthless dictator, which I’m not sure if has).
It may ultimately come down to anti-aging technology ultimately come at just the right time to save us from the worst of authoritarianism (given that we no longer have Stalin or Mao).
2021 edit: Though who knows, democracies can easily turn into authoritarian regimes, and all it takes is a single terrorist or bioterrorist attack that forces universal surveillance...
As I wrote here, I think this could be due (in part) to biases accumulated by being in a field (and being alive) longer, not necessarily (just) brain aging. I’d guess that more neuroplasticity or neurogenesis is better than less, but I don’t think it’s the whole problem. You’d need people to lose strong connections, to “forget” more often.
George Church is over 60 and I’ve heard some people refer to him as a “child”, given that he seems to not strongly identify with strongly held beliefs or connections (he’s also not especially attached to a certain identity). I talked to him—he cares more about regeneration/rejuvenation—or maintaining the continuity of consciousness and the basic gist of his personality/mode of being than about maintaining specific memories (regeneration/rejuvenation research may ultimately come down to replacing old parts of your brain or identity with new untrained tissue—this is where developmental biology/SCRB becomes especially relevant). In fact, he’s unironically bullish about anti-aging therapies coming in his lifetime
Besides the cancer thing, SENS ignores telomere attrition, because it’s still unclear if telomere attrition is a significant cause of aging. And the likelihood that WILT will be needed is still above 50%.
Isn’t early detection of cancer (and intervention) more feasible?
Categorizing one’s favorites (or putting them in folders) so one doesn’t have to scroll through them all to the beginning.