I do independent research on EA topics. I write about whatever seems important, tractable, and interesting (to me). Lately, I mainly write about EA investing strategy, but my attention span is too short to pick just one topic.
I have a website: https://mdickens.me/ Most of the content on my website gets cross-posted to the EA Forum.
My favorite things that I’ve written: https://mdickens.me/favorite-posts/
I used to work as a software developer at Affirm.
Vasco’s corpus of cost-effectiveness estimates
Are you talking about this post? Looks like those cost-effectiveness estimates were written by Ambitious Impact so I don’t know if there are some other estimates written by Vasco.
I do think there’s concern with a popular movement that the movement will move in a direction you didn’t want, but empirically this has already happened for “behind closed doors” lobbying so I don’t think a popular movement can do worse.
There’s also an argument that a popular movement would be too anti-AI and end up excessively delaying a post-AGI utopia, but I discussed in my post why I don’t think that’s a sufficiently big concern.
(I agree with you, I’m just anticipating some likely counter-arguments)
Quick thoughts on investing for transformative AI (TAI)
Some EAs/AI safety folks invest in securities that they expect to go up if TAI happens. I rarely see discussion of the future scenarios where it makes sense to invest for TAI, so I want to do that.
My thoughts aren’t very good, but I’ve been sitting on a draft for three years hoping I develop some better thoughts and that hasn’t happened, so I’m just going to publish what I have. (If I wait another 3 years, we might have AGI already!)
Scenarios where investing doesn’t work:
Takeoff happens faster than markets can react, or takeoff happens slowly but is never correctly priced in.
Investment returns can’t be spent fast enough to prevent extinction.
TAI creates post-scarcity utopia where money is irrelevant.
It turns out TAI was already correctly priced in.
Scenarios where investing works:
Slow takeoff, market correctly anticipates TAI after we do but before it actually happens, and there’s a long enough time gap that we can productively spend the earnings on AI safety.
TAI is generally good, but money still has value and there are still a lot of problems in the world that can be fixed with money.
(Money seems much more valuable in scenario #5 than #6.)
What is the probability that we end up in a world where investing for TAI turns out to work? I don’t think it’s all that high (maybe 25%, although I haven’t thought seriously about this).
You also need to be correct about your investing thesis, which is hard. Markets are famously hard to beat.
Hardware makers (e.g. NVIDIA)? Anecdotally this seems to be the most popular thesis. This is the most straightforward idea but I am suspicious that a lot of EA support for investing in AI looks basically indistinguishable from typical hype-chasing retail investor behavior. NVIDIA already has a P/E of 56. There is a 3x levered long NVIDIA ETP. That is not the sort of thing you see when an industry is overlooked. Not to say NVIDIA is definitely a bad investment, it could be even more valuable than the market already thinks, I’m just wary.
AI companies? This doesn’t seem to be a popular strategy, the argument against is that it’s a crowded space with a lot of competition which will drive margins down. (Whereas NVIDIA has a ~monopoly on AI chips.) Plus I am concerned that giving more money to AI companies will accelerate AI development.
Energy companies? It’s looking like AI will consume quite a lot of energy. But it’s not clear that AI will make a noticeable dent on global energy consumption. This is probably the sort of thing you could make reasonable projections for.
Out-of-the-money call options on a broad index (e.g. S&P 500 or NASDAQ)? This strategy avoids making a bet about which particular companies will do well, just that something will do much better than the market anticipates. But I’d also expect that unusually high market returns won’t start showing up until TAI is close (even in a slow-takeoff world), so you have less time to use the extra returns to prevent AI-driven extinction.
Commodities? The idea is that anything complicated will become much easier to produce thanks to AI, but commodities won’t be much easier to get, so their prices will go up a lot. This is an interesting idea that I heard recently, I have no idea if it’s correct.
Momentum funds (e.g. VFMO or QMOM)? The general theory of momentum investing is that the market under-reacts to slow news. The pro of this strategy is that it should work no matter which stocks/industries benefit from AI. The con is that it’s slower—you don’t buy into a stock until it’s already started going up. (I own both VFMO and QMOM (mostly QMOM), a bit because of AI but mainly because I think momentum is a good idea in general.)
The way they’re usually done, awards counteract the negative:positive feedback ratio for a tiny group of people. I think it would be better to give positive feedback to a much larger group of people, but I don’t have any good ideas about how to do that. Maybe just give a lot of awards?
I think that such an obviously fraught and tense issue deserves more thought and care than a quick BOTEC.
I am opposed to adding more barriers to doing BOTECs, they’re already difficult enough and rare enough as it is. I appreciate that OP did a BOTEC.
I spent some time looking into this since it was not obvious to me how to buy from Perfect Day. Looks like the only retail partner who sells their whey protein powder is Myprotein, most retailers sell things like ice cream.
Isn’t it one of the cheapest forms of protein?
But these should be matched by looking for cases where something good happened because people tried to accumulate power/influence within a system.
I think this is a significant percent of all good things that have ever happened.
I think you are right about this, you’ve changed my mind (toward greater uncertainty).
My gut feeling is that [...] the biggest difference between good outcomes and bad outcomes is how much work the big AI labs put into alignment during the middle of the intelligence explosion when progress moves fastest.
This seems to depend on a conjunction of several strong assumptions: (1) AI alignment is basically easy; (2) there will be a slow takeoff; (3) the people running AI companies are open to persuasion, and “make AI safety seem cool” is the best kind of persuasion.
But then again I don’t think pause protests are going to work, I’m just trying to pick whichever bad plan seems the least bad.
A time cost of 0.0417 $/d for 7.5 s/d and 20 $/h.
Nitpick: I just timed myself taking creatine and it took me 42 seconds.
(My process consists of: take creatine and glass out of cabinet; scoop creatine into glass; pour tap water into glass; drink glass; put creatine and glass back into cabinet.)
Agreed that creatine passes a cost-benefit analysis.
I think this is the best explanation I’ve seen, it sounds likely to be correct.
That sounds pretty reasonable for why psychotherapy wouldn’t be as widespread as it should. It looks to me like most of these reasons wouldn’t apply to AMF. Training new psychotherapists takes years and tens of thousands of dollars (at developing-world wages). Getting more malaria nets requires buying more $5 malaria nets, and distributing malaria nets is much easier than distributing psychotherapists. So reasons 1–3 and #6 don’t carry over (or at least not to nearly the same extent). #4 doesn’t seem relevant to my original question so I think #5 is the only one that carries over—recipients might not know that they should be concerned about malaria.
Why does distributing malaria nets work? Why hasn’t everyone bought a bednet already?
If it’s because they can’t afford bednets, why don’t more GiveDirectly recipients buy them?
Is it because nobody in the local area sells bednets? If so, why doesn’t anyone sell them?
Is it because people don’t think bednets are worth it? If so, why do they use the bednets when given them for free?
I don’t know why (I thought it was a good post) but I have some guesses:
Maybe people don’t like the framing of o1 as “lying” when it’s not clear that it lied. All that’s clear is that o1 gave a false justification, which isn’t necessarily lying.
You tend to write in an alarmist rhetorical style that I think turns off a lot of people. I think you are very much correct to be alarmed about AI x-risk, but also I don’t think it’s a good persuasive strategy (for an EA/rationalist audience) to convey this through emotionally charged rhetoric. I didn’t think this particular post was alarmist, but you have a history of writing alarmist posts/comments, so maybe people downvoted based on the title.
$100 million/year and 600 people = $167,000 per person-year
$1M buys 10 person-years = $100,000 per person-year
These numbers are approximately the same. I don’t understand how you get that 5⁄6 of the work comes from volunteering / voluntary underpayment, did I do it wrong?
I plan on donating to PauseAI, but I’ve put considerable thought into reasons not to donate.
I gave some arguments against slowing AI development (plus why I disagree with them) in this section of my recent post, so I won’t repeat those.
There’s not that much evidence that protests are effective. There’s some evidence, including a few real-world natural experiments and some lab experiments, but this sort of research doesn’t have a good replication record.
Research generally suggests that peaceful protests work, while violent protests reduce public support. If violent protests backfire, maybe some types of peaceful protests also backfire. And we don’t have enough granularity in the research to identify under what specific conditions peaceful protests work, so maybe PauseAI-style protests will backfire.
Polls suggest there is broad public support for a pause. If there’s already public support, doesn’t that weaken the case for protesting? Perhaps that means we should be doing something else instead.
Protesting might not be a good use of resources. It might be better to lobby policy-makers.
Yes that’s also fair. Conflicts of interest are a serious concern and this might partially explain why big funders generally don’t support efforts to pause AI development.
I think it’s ok to invest a little bit into public AI companies, but not so much that you’d care if those companies took a hit due to stricter regulations etc.
I think the position I’m arguing for is basically the standard position among AI safety advocates so I haven’t really scrutinized it. But basically, (many) animals evolved to experience happiness because it was evolutionarily useful to do so. AIs are not evolved so it seems likely that by default, they would not be capable of experiencing happiness. This could be wrong—it might be that happiness is a byproduct of some sort of information processing, and sufficiently complex reinforcement learning agents necessarily experience happiness (or something like that).
Also: According to the standard story where an unaligned AI has some optimization target and then kills all humans in the interest of pursuing that target (e.g. a paperclip maximizer), it seems unlikely that this AI would experience much happiness (granting that it’s capable of happiness) because its own happiness is not the optimization target.
(Note: I realize I am ignoring some parts of your comment, I’m intentionally only responding to the central point so my response doesn’t get too frayed.)
I want to add that I think the argument you present here is better than any of the arguments I’d considered in the relevant section.
I think it’s quite unlikely that a misaligned AI would create an AI utopia. Much more likely that it would create something that resembles a paperclip maximizer / something that has either no conscious experience, or experiences with no valance, or experiences with random valences.
I think this is the comment you’re thinking of: https://forum.effectivealtruism.org/posts/jAfhxWSzsw4pLypRt/where-i-am-donating-in-2024?commentId=y6evejGaj4TMApozC