I saw your request for commentary on Facebook, so here are some off-the-cuff comments (about 1 hour’s worth so take with appropriate grains of salt, but summarizing prior thinking):
My prior take on metformin was that it seems promising for its space (albeit with mixed evidence, and prior longevity drug development efforts haven’t panned out, but the returns would be very high for medical research if true), although overall the space looks less promising than x-risk reduction to me; the following comments will be about details of the analysis where I would currently differ
The suggestion of this trial moving forward LEV by 3+ years through an icebreaker effect boosting research looks wildly implausible to me
LEV is not mainly bottlenecked on ‘research on aging,’ e.g. de Grey’s proposals require radical advances in generally medically applicable stem cell and genetic engineering technologies that already receive massive funding and are quite challenging; the ability to replace diseased cells with genetically engineered stem cell derived tissues is already a major priority, and curing cancer is a small subset of SENS
Much of the expected gain in biomedical technology is not driven by shifts within biology, and advances within a particular medical field are heavily driven by broader improvements (e.g. computers, CRISPR, genome sequencing, PCR, etc); if LEV is far off and heavily dependent on other areas, then developments in other fields will make it comparatively easy for aging research to benefit from ‘catch up growth’ reducing the expected value of immediate speedup (almost all of which would have washed away if LEV happens in the latter half of the century)
In particular, if automating R&D with AI is easier than LEV, and would moot prior biomedical research, then that adds an additional discount factor; I would bet that this happens before LEV through biomedical research
Getting approval to treat ‘aging’ isn’t actually particularly helpful relative to approval for ‘diseases of aging’ since all-cause mortality requires larger trials and we don’t have great aging biomarkers; and the NIH has taken steps in that direction regardless
Similar stories have been told about other developments and experiments, which haven’t had massive icebreaker effects
Combined, these effects look like they cost a couple orders of magnitude
From my current epistemic state the expected # of years added by metformin looks too high
Re the Guesstimate model the statistical power of the trial is tightly tied to effect size; the larger the effect size the fewer people you need to show results; that raises the returns of small trials, but means you have diminishing returns for larger ones (you are spending more money to detect smaller effects so marginal cost-effectiveness goes a lot lower than average cost-effectiveness, reflecting high VOI of testing the more extravagant possibility)
Likewise the proportion using metformin conditional on a positive result is also correlated with effect size (which raises average EV, but shifts marginal EV lower proportionate to average EV); also the proportion of users seems too low to me conditional on success
Hey! Thanks for the comment! I really appreciate it. For some reason I’m only seeing it now and by chance. I don’t know if I didn’t get the notification or if I missed it.
Regarding the content of your comment: I agree with most of it. In fact 3.6 years is probably a big overestimation. However, I still think, in general, that bringing LEV forward could be a big contributor to the cost-effectiveness of aging research in general. In my newer post I lay your same arguments about improving technology that may subsume the effect of today’s research, making it less cost-effective. This factor also influences variable E in the TAME analysis, which I also probably vastly overestimated. For the very specific case of AI potentially automating R&D I think the timeline is longer than for LEV achieved through biomedical research (I’m taking the view that arises from the probability distribution given by AI researchers), but, as you said, it’s not the only technology that would make some of the efforts made now less useful.
Maybe I’m still less “pessimistic” than you in the sense that I think that an ice-breaking effect could enable more research on what are neglected facets of aging for which treatments could be devised much more quickly. The foundational research is not very neglected, while there are wide areas of translational research that could use much more funding and that are necessary to reach the final goal. From the lifespan.io’s Rejuvenation Roadmap you should get a preliminary idea.
Your example using the SENS approach is correct: areas like stem cell research and cancer research don’t seem to be underfunded. But they are only two pieces of the puzzle. Some others are being much more neglected. That’s why SENS itself gives higher priority to the most neglected areas, like mitochondrial dysfunction and crosslinks, which should be also more tractable (an interesting fact is that Aubrey de Grey often emphasises neglectedness, tractability and scope in his conferences, but I haven’t heard anyone within EA pointing this out). If stem cells research, cancer and other difficult and highly funded areas were all there is to aging research, it wouldn’t look like a very good candidate EA cause. In fact, not only de Grey but many researchers in the area are pursuing projects they believe are very much funding constrained (example: Steve Horvath).
About the comparison with x-risk reduction: Yes, I broadly agree that x-risk reduction looks overall more promising as a cause area. However I think that many x-risk focused interventions have a higher level of uncertainty. It also seems that within Effective Altruism little to no effort has been made to evaluate aging research, while, to me, it looks highly competitive with many of the other focuses of EAs (some specific interventions inside aging research should be very recognisably better). So it should be analysed further, especially because we may be missing out on especially important opportunities.
Regarding the expected number of years added by metformin: I think “one year” is a very conservative number given the evidence I’ve presented, and you’ll often hear researchers estimating more.
Hi Emanuele,
I saw your request for commentary on Facebook, so here are some off-the-cuff comments (about 1 hour’s worth so take with appropriate grains of salt, but summarizing prior thinking):
My prior take on metformin was that it seems promising for its space (albeit with mixed evidence, and prior longevity drug development efforts haven’t panned out, but the returns would be very high for medical research if true), although overall the space looks less promising than x-risk reduction to me; the following comments will be about details of the analysis where I would currently differ
The suggestion of this trial moving forward LEV by 3+ years through an icebreaker effect boosting research looks wildly implausible to me
LEV is not mainly bottlenecked on ‘research on aging,’ e.g. de Grey’s proposals require radical advances in generally medically applicable stem cell and genetic engineering technologies that already receive massive funding and are quite challenging; the ability to replace diseased cells with genetically engineered stem cell derived tissues is already a major priority, and curing cancer is a small subset of SENS
Much of the expected gain in biomedical technology is not driven by shifts within biology, and advances within a particular medical field are heavily driven by broader improvements (e.g. computers, CRISPR, genome sequencing, PCR, etc); if LEV is far off and heavily dependent on other areas, then developments in other fields will make it comparatively easy for aging research to benefit from ‘catch up growth’ reducing the expected value of immediate speedup (almost all of which would have washed away if LEV happens in the latter half of the century)
In particular, if automating R&D with AI is easier than LEV, and would moot prior biomedical research, then that adds an additional discount factor; I would bet that this happens before LEV through biomedical research
Getting approval to treat ‘aging’ isn’t actually particularly helpful relative to approval for ‘diseases of aging’ since all-cause mortality requires larger trials and we don’t have great aging biomarkers; and the NIH has taken steps in that direction regardless
Similar stories have been told about other developments and experiments, which haven’t had massive icebreaker effects
Combined, these effects look like they cost a couple orders of magnitude
From my current epistemic state the expected # of years added by metformin looks too high
Re the Guesstimate model the statistical power of the trial is tightly tied to effect size; the larger the effect size the fewer people you need to show results; that raises the returns of small trials, but means you have diminishing returns for larger ones (you are spending more money to detect smaller effects so marginal cost-effectiveness goes a lot lower than average cost-effectiveness, reflecting high VOI of testing the more extravagant possibility)
Likewise the proportion using metformin conditional on a positive result is also correlated with effect size (which raises average EV, but shifts marginal EV lower proportionate to average EV); also the proportion of users seems too low to me conditional on success
Hey! Thanks for the comment! I really appreciate it. For some reason I’m only seeing it now and by chance. I don’t know if I didn’t get the notification or if I missed it.
I’m not sure if this is the post I was asking feedback for though. This analysis is from nine months ago, and my views on it changed. On Facebook I was probably referring to this other post I made recently: A general framework for evaluating aging research. Part 1: reasoning with Longevity Escape Velocity. [EDIT: I just saw you made a comment under that post too, so never mind].
Regarding the content of your comment: I agree with most of it. In fact 3.6 years is probably a big overestimation. However, I still think, in general, that bringing LEV forward could be a big contributor to the cost-effectiveness of aging research in general. In my newer post I lay your same arguments about improving technology that may subsume the effect of today’s research, making it less cost-effective. This factor also influences variable E in the TAME analysis, which I also probably vastly overestimated. For the very specific case of AI potentially automating R&D I think the timeline is longer than for LEV achieved through biomedical research (I’m taking the view that arises from the probability distribution given by AI researchers), but, as you said, it’s not the only technology that would make some of the efforts made now less useful.
Maybe I’m still less “pessimistic” than you in the sense that I think that an ice-breaking effect could enable more research on what are neglected facets of aging for which treatments could be devised much more quickly. The foundational research is not very neglected, while there are wide areas of translational research that could use much more funding and that are necessary to reach the final goal. From the lifespan.io’s Rejuvenation Roadmap you should get a preliminary idea.
Your example using the SENS approach is correct: areas like stem cell research and cancer research don’t seem to be underfunded. But they are only two pieces of the puzzle. Some others are being much more neglected. That’s why SENS itself gives higher priority to the most neglected areas, like mitochondrial dysfunction and crosslinks, which should be also more tractable (an interesting fact is that Aubrey de Grey often emphasises neglectedness, tractability and scope in his conferences, but I haven’t heard anyone within EA pointing this out). If stem cells research, cancer and other difficult and highly funded areas were all there is to aging research, it wouldn’t look like a very good candidate EA cause. In fact, not only de Grey but many researchers in the area are pursuing projects they believe are very much funding constrained (example: Steve Horvath).
About the comparison with x-risk reduction: Yes, I broadly agree that x-risk reduction looks overall more promising as a cause area. However I think that many x-risk focused interventions have a higher level of uncertainty. It also seems that within Effective Altruism little to no effort has been made to evaluate aging research, while, to me, it looks highly competitive with many of the other focuses of EAs (some specific interventions inside aging research should be very recognisably better). So it should be analysed further, especially because we may be missing out on especially important opportunities.
Regarding the expected number of years added by metformin: I think “one year” is a very conservative number given the evidence I’ve presented, and you’ll often hear researchers estimating more.