Hi there! Iām an ML research scientist at Upstart and am currently earning to give. I have some prior experience in the economics of AI and dabbling in AI technical safety research.
CalebMarescašø
Karma: 140
Honestly, I fail to see how this responds to my comment. I will try to explain again.
There are two arguments here. My argument was that a profit-maximizing firm will invest less in improving taste than one that internalizes the externalities of replacing meat with plant-based foods, or whose customers internalize those externalities. Similarly, Apple has an incentive to increase the quality of the iPhone, but its incentive is tempered by cost. It chooses the quality that maximizes profit, not the maximum quality possible. Maybe for every GB of storage they cram into the iPhone, a baby alien is saved from a terrible death on a faraway planet. But if neither Apple nor Appleās customers internalize those externalities, Apple is going to put fewer GB of storage into the iPhone than is socially optimal.Here is maybe a better example: if we found out that adding extra GBs of storage to iPhones was somehow a massive boon for insect welfare, do we think that Apple would seriously put much effort into maximizing the number of GBs of storage in iPhones? No. Very few people care about insect welfare, and Apple would continue to market iPhones with the same amount of storage as they had previously planned. Even if Apple did care, if its customers didnāt, then Apple might struggle to raise enough capital from profit-maximizing investors to reach the socially optimal level of GB of storage. (Ok, Apple would not struggle to raise capital, but Beyond Food might!)
The second argument, and this is the argument directly made in the RFP, is that they plan to fund foundational pre-competitive research questions. This is a well-known market failure in economics. Foundational research is often non-excludable and non-rival. That is, you canāt patent it, and my use of it doesnāt diminish your ability to use it. In short, a firm doing this research would absorb all of the cost but would not capture all of the benefit. This is why foundational research is often funded by governments and philanthropic organizations.Maybe there are reasons why these arguments donāt work in this case! But I would like to see counterarguments. Simply claiming that incentives are aligned doesnāt address either of these arguments.
Letās assume for simplicity that these companies are profit-maximizing. Then they are, in fact, not incentivised to maximize tastiness. Taste can increase revenue, but it also increases cost. Therefore, the profit-maximizing investment in taste-improving research need not equal the socially optimal investment.
But actually, that is not the argument used in the post. They say:These are foundational, pre-competitive research problems. They are unlikely to be solved by any single company and are systematically underfunded by both public agencies and private R&D budgets, making them strong candidates for philanthropic support.
So this is a classic underprovision of fundamental research issue.
Saving animals from suffering in factory farms is an externality not priced in by the market. Therefore, the optimal investment in alternative meat is greater than the profit-maximizing investment.
Hi Garrison,
I notice that only the paperback is available for pre-order on Amazon. Will there be a kindle version?
Hi Jason, thanks for this. I was not aware of this review article. There is a new review article that came out this year, which concludes that there is insufficient evidentiary basis for harm from high protein intake. In particular, it seems like some of the results of previous studies may have been confounded with calorie intake.
Iām not a nutritionist or an exercise scientist, so I could be interpreting this incorrectly, but I think you are overly dismissive of the idea that people should be eating more protein.
The guidelineās recommendation of 0.7 to 1 gram per kilogram daily represents the minimum intake needed to prevent malnutrition and maintain nitrogen balance; it is insufficient for optimal muscle growth when combined with strength training.[1] For people who are trying to increase their muscle mass, Hubermanās suggestion is accurate and helpful.[2] Since muscle mass is associated with lower all-cause mortality, I think that Hubermanās suggestion (when accompanied with adequate exercise) is more beneficial than the government guideline. This is especially true for older adults, but to ensure healthy aging and longer health spans, it is preferable to build muscle throughout adulthood.
That said, we should, of course, be encouraging people to get as much of their protein intake as possible from non-animal sources. Personally, I encourage vegans to exercise and try to eat in the range of 0.7 to 1 gram of protein per pound (1.5 to 2.2 grams per kg) of body weight per day, not only for your own health, but to be an example to others that they can reduce animal suffering without sacrificing their health or muscle growth.- ^
Note that the review you linked specifically excludes multi-component interventions, including protein and exercise combinations.
- ^
https://āābjsm.bmj.com/āācontent/āā52/āā6/āā376
https://āāpmc.ncbi.nlm.nih.gov/āāarticles/āāPMC8978023/āā
Note that the first meta-analysis finds that the effect of supplementing protein plateaus at 1.6 g/ākg. This is still within Hubermanās range, and is still much higher than the guideline of 0.7 ā 1. Iāve seen people in the exercise community quibble a lot about this supposed plateau. My position is that in expectation the benefit of going up from 1.6 g/ākg to 2.2 g/ākg is higher than the (pretty much non-existent) risk.
- ^
Now, this isnāt the best way to maximize expected value. If you were an expected value maximizing robot, you would not pursue this strategy. You would say ābleep bloop, this brings about 93.5 fewer expected utils than the other strategy.ā But I assume you are not an EV maximizing robot.
Hmmm. This is interesting, as diversification is expected utility maximizing in the finance context. The fact that it is not EV maximizing in the utilitarian framework makes me wonder if there is something wrong with the framing.
The obvious difference is that EV is risk-neutral. I think this is usually justified by the fact that we are counting utils, whereas in finance, we are counting dollars. Arguably, it makes sense that utility is concave in dollars, but not that utility is (strictly) concave in itself.
Intuitively, this seems wrong to me. Imagine a choice set {A, B}. Choice A has a 50% chance of resulting in a world with 1 million sentient beings, all with 100 utils each, and a 50% chance of resulting in a world with zero sentient beings. Choice B results with certainty in a world with 500 thousand sentient beings, all with 100 utils each. I strictly prefer Choice B to Choice A, implying that I have a moral meta utility function that is concave in utils. This is sufficient to derive that diversification is optimal.
Thanks for this Phil,
A couple of questions regarding SWE:
āSecond-wave endogenousā (Iāll write āSWEā) growth models posit instead that technology grows exponentially with a constant or with a growing population. The idea is that process efficiencyāthe quantity of a given good producible with given labor and/āor capital inputsāgrows exponentially with constant research effort, as in a first-wave endogenous model; but when population grows, we develop more goods, leaving research effort per good fixed.
Only one SWE model avoids a conclusion along these lines: the first one, Young (1998). It avoids the conclusion by positing that vertical innovation faces severely diminishing returns to research labor, so that in effect, a larger population can only be employed productively in research if the range of goods widens.
So does this mean that in the research production function, the exponent on the stock of āideasā is one, and the exponent on the number of researchers is significantly less than one? It might be nice to see the equation.
Relatedly, isnāt endogenous growth a knife-edge case? Intuitively, it seems unlikely to be true, and SWE doesnāt seem to address this issue.
Iām imagining something that is Cobb-Douglas between capital and land. Growth should be exponential (not super exponential) when A_auto is growing at a constant rate, same as a regular Cobb-Douglas production function between capital and labor. Specifically, I was thinking something like this:
X_old^beta(A_old K_old^alpha L^{1-alpha})^(1-beta) + X_auto^beta(A_auto K_auto)^(1-beta)
st X_old + X_auto = X_total (allocating land between the two production technologies)
As to your second point, yes, you are correct, as long as A_old is constant wages would not increase.
Thanks! Looks like I have access through my university.
Diverging utilities can be an issue. You can also get infinite output in finite time. The larger issue is that the economy has no steady state. In economic growth models, a steady state (or balanced growth path (BGP)) represents a long-run equilibrium where key economic variables per capita (like capital per worker, output per worker, consumption per worker) grow at constant rates. This greatly simplifies the analysis.
For example, I have a paper in which I analyze how households would behave if they expected TAI to transform the economy. To do this, I calculated the steady state the economy was in prior to households learning about the potential of TAI as well as the post-TAI steady state. I could then calculate the transition path between these two steady states. I thought of using the same production function you used here, but then there wouldnāt have been a post-TAI steady state, which is necessary to be able to find the transition path.
A production function that I have mused about is one like yours, but with land. This should solve the issue, as the post-TAI economy will no longer be AK. It also addresses another issue I have with growth models: itās really tricky to get wages to decrease in the long run. For the production function you use, if A_{old} was also growing at a constant rate, then wages would eventually rise. This is because the new production technology doesnāt harm the old technology other than by taking capital away. Each production technology could work next to the other without interfering. In reality, there is a limited amount of space on earth and once the new technology is more efficient, it isnāt profit maximizing to āwaste spaceā on the old style of production. I donāt know if it is worth modeling for what you are doing, might be too many bells and whistles, but it is something Iāve been thinking about.
Thanks for this. If I understand correctly, the result is primarily driven by the elastic labor supply, which is a function of W and not of R, and the constant supply of capital. This seems most relevant for very fast takeoff scenarios.
My intuition is that as people realize that their jobs are being automated away, they will want to work more to bolster their savings before we move into the new regime where their labor is worthless and capital is all that matters. This would require fully modeling the householdās intertemporal utility and endogenizing the capital supply. This might be tricky, however, with your production function, because if A_{auto} is increasing at a constant rate and households are allowed to save, you will get superexponential growth.
Thanks for this excellent primer and case study! I learned a lot about causal analysis from your explanation. The section on using three waves to control for confounders while avoiding controlling for potential mediators was particularly helpful. I would be interested in hearing more about how the sensitivity analysis for unmeasured confounders works.
The positive effect of activism on meat consumption that you found is especially concerning and important. I hope that we can gain more insight into this soon. If this finding replicates, then a lot of organizations might have to reevaluate their methods.
Hi Matthew,
Thank you for your comment. I think this is a reasonable criticism! There is definitely an endogenous link between investment and AI timelines that this model misses. I think that this might be hard to model in a realistic way, but I encourage people to try!
On the other hand, I think the strategic motivation is important as well. For example, here is Satya Nadella on the Dwarkesh Podcast:And by the way, one of the things is that there will be overbuild. To your point about what happened in the dotcom era, the memo has gone out that, hey, you know, you need more energy, and you need more compute. Thank God for it. So, everybodyās going to race.
In reality, both mechanisms are probably in play. My paper is intended to focus on the race mechanism.
Two more notes: higher savings imply lower consumption in the short term. However, even if TAI isnāt invented, consumption will rise higher than in the stationary equilibrium purely from capital accumulation.
Lastly, the main thrust of the paper is on the implications for interest rates, I do not intend to make strong claims about social welfare.
I donāt think that the possible outcomes of AGI/āsuperintelligence are necessarily so binary. For example, I am concerned that AI could displace almost all human labor, making traditional capital more important as human capital becomes almost worthless. This could exacerbate wealth inequality and significantly decrease economic mobility, making post-AGI wealth mostly a function of how much wealth you had pre-AGI.
In this scenario, saving more now would enable you to have more capital while returns to capital are increasing. At the same time, there could be billions of people out of work without significant savings and in need of assistance.
I also think even if AGI goes well for humans, that doesnāt necessarily translate into going well for animals. Animal welfare could still be a significant cause area in a post-AGI future and by saving more now, you would have more to donate then (potentially a lot more if returns to capital are high).
Why would Knightian uncertainty be an argument against AI as an existential risk? If anything, our deep uncertainty about the possible outcomes of AI should lead us to be even more careful.
Similar campaigns have worked really well for animal advocacy, so Iām excited to see what you can accomplish.
Iām wondering, what kinds of tasks can volunteers help with? If I have no social media accounts or experience trying to promote causes on social media is there anything I can do?
However, if they believe in near-term TAI, savvy investors wonāt value future profits (since theyāll be dead or super rich anyways)
My future profits arenāt very relevant if Iām dead, but I might still care about it even if Iām super rich. Sure, my marginal utility will be very low, but on the other hand the profit from my investments will be very large. Even if everyone is stupendously rich by todayās terms, there might be a tangible difference between having a trillion dollars in your bank account and having a quadrillion dollars in your bank account. Maybe I want my own galaxy in which I alone have the rights to build Dyson spheres and that is out of the price range of your average joe with a trillion-dollar net wealth. Maybe (and this might be more salient to your typical investor who isnāt actively thinking about far out sci-fi scenarios) I want the prestige, political control, etc, that come with being wealthy compared to everyone else.
A bet that interest rates will rise is not a bet on short AI timelines. Rather, it is a bet that:
Most consumers will correctly perceive that AI timelines are short, and
Most consumers will realize this long enough before TAI that there is enough time to benefit from profitable bets made now, and
Most consumers will believe that transformative AI will significantly reduce the marginal utility they get from their savingsāand not, say, increase the marginal value of saving, because they could lose their jobs without taking part in the newfound prosperity from AI
I believe that this is almost correct. My objection is with the second bullet point, āinterest rates can rise before we get TAIā. This is possible, but we no longer have a reason to believe that it will happenāunless very many people decide to reduce their savings rates. By then, this is no longer a bet on short AI timelines, but rather a bet about whether the typical consumer will realize that AI timelines are short sufficiently long enough before AI that you have time to enjoy your profits.
If future benefits exist for being even richer after TAI, interest rates could rise due to inductive reasoning even before consumers begin adjusting their savings rates in response to TAI. If I know that consumers will adjust their savings rate one day before TAI (assuming a deterministic timeline where TAI occurs in one discontinuous jump and very unrealistic timescales for consumers changing their savings rate for simplicityās sake), then I should place a bet on the interest rate rising (e.g. shorting government bonds) two days before TAI. If enough investors take this action, then interest rates will rise two days before TAI. Knowing this, I should short government bonds three days before TAI, etcā¦ Similar to how if the government promises to print a lot of money in one month, then inflation will begin to rise immediately.
I am not aware of any international treaties which sanction the use of force against a non-signatory nation except for those circumstances under which one of the signatory nations is first attacked by a non-signatory nation (e.g. collective defense agreements such as NATO). Your counterexample of the Israeli airstrike on the Osirak reactor is not a precedent as it was not a lawful use of force according to international law and was not sanctioned by any treaty. I agree that the Israeli government made the right decision in orchestrating the attack, but it is important to point out the differences between that and what you are suggesting.
Ultimately, to quibble about whether your suggestion is an āact of violenceā or not misses the point. What you suggest would be an unprecedented sanctioning of force. I believe the introduction of such an agreement would be very incendiary and would offer a bad precedent. Note that no such agreement was signed in order to prevent nuclear proliferation. Many experts were very worried that nuclear weapons would proliferate much further than they ultimately did. Sometimes the use of force was used, but always with a lighter hand than āletās sign a treaty to bomb anyone we think has a reactor.ā
There is definitely a miscommunication occurring here. I am not arguing that improving taste has intrinsic social value, nor does my argument hinge on it. Quite the opposite, argument 1 requires an externality beyond taste. To state clearly, my arguments show that there is a market failure that causes the profit-maximizing investment in taste to be lower than the socially optimal investment in taste, where the socially optimal level is higher since higher tastiness would cause increased consumption, which would lower consumption of animal products, which would alleviate animal suffering.
To clarify the example in argument 1, I meant to say every iPhone sold, not every iPhone produced. Note that the causal chain is the same. This is precisely why I chose this example.
More storage ā more people buy iPhones ā better insect welfare = win
Tastier meat alternatives ā> more people eat meat alternatives ā> less animal suffering = win
The example is not perfect, because tastier meat alternatives only affect animal suffering through the number of meat alternatives sold (and more precisely, the number of animal products displaced), whereas, in the iPhone example, there is the extrinsic margin of increasing the number of iPhones sold, and also the intrinsic margin of increasing the number of GB, conditional on sale. I can see why this would be confusing and make it sound like the taste of the meat alternatives is presumed to be intrinsically good. That was not my intention. I intended to highlight a product with a positive externality, where increasing its value to the customer would also create additional social value.
To that end, letās fix the example. Letās assume that we found out that every iPhone sold was somehow a massive boon for insect welfare. The social value of increasing the number of GB of storage in iPhones would increase upon the news, as increasing the storage capacity would make the iPhone more attractive to consumers, which would increase the number of iPhones sold, which would lead to better insect welfare. Likewise, any increase in the quality of iPhones would have the same effect. Storage was arbitrarily chosen as a metric that people care about, like the taste of food. However, I highly doubt that Apple would do much at all, if anything, to increase the number of iPhones sold upon learning this news regarding insect welfare.
Thus, we can see that the introduction of an externality to the number of iPhones sold led to underprovisioning of iPhone quality-improvement research relative to what is socially optimal (profit-maximizing quality-improvement research remained the same, whereas the socially optimal value increased).
Regarding foundational research, the fact that food manufacturers invest in taste research does not imply a lack of underprovision of foundational research. Pretty much all industries do research. Foundational research is not the same as other kinds of research. Since this market failure is well-known, I donāt think it is worth hashing out in detail, though I did explain it briefly above. You could argue that there isnāt any foundational research to do in this field, or that it is very intractable, but arguing that a well-known market failure doesnāt exist will be difficult.