I’m not clear on whether you think the drowning child argument is browbeating by nature, or whether you think that just this particular presentation of it is browbeating. (Your remark about retiring the drowning child implies the former, but another of your comments elsewhere implies that you can use the drowning child argument without browbeating people with it?)
Anyway, I don’t think it’s time to retire the argument, I still feel like I hear a lot of people cite it as insightful for them.
I think the word “lock-in” can be confusing here. I usually think of “lock-in” as worrying about a future where things stop improving, or a particular value system or set of goals gets permanent supremacy. If this is what we mean, then I don’t think “the future is out of human hands” is a sufficient for lock-in, because the future could continue to be dynamic or uncertain or getting better or worse, with AIs facing new and unique challenges and rising to them or failing to rise to them. Whatever story humans have set in motion is “locked in” in the sense that we can no longer influence it, but not in the sense that it’ll necessarily have a stable state of affairs persist for those who exist in it. Maybe it’s clearer to think of humans being “locked out” here, while AIs continue to have influence.
I should also say I’m grateful to you for what is obviously a pretty significant amount of work, done thoughtfully and with measured conclusions.
(also I think there are some links missing from the Recommendations section?)
FWIW it strikes me as odd / surprising to say that e.g. Black African Americans are not Westerners purely by virtue of not being white or mixed race.
this investigation would bring that closure to the rest of EA.
I think how much closure the investigation brings will depend significantly on what it includes and what it concludes, and I think different people will have different standards about what will satisfy them. While I am in favour of more investigation, I would guess that realistically feasible investigations will not be able to close all relevant questions or really settle everything relevant in the collective mind of the community.
All of the footnote links are broken for me, both from the text to the footnote and the footnote back to the text :(
As a software developer looking for projects:
It seems like a lot of careful thought went into the overall design and specification of the tool. How much and what kind of software work was required to turn it into a reality? Do you have other plans for projects like this that are bottlenecked on software capacity?
Have you considered open-sourcing the project?
I suggest editing in a link to the LessWrong linkpost
Assessments of non-AI x-risk are relevant to AI safety discussions because some of the hesitance to pause or slow AI progress is driven by a belief that it will help eliminate other threats if it goes well.
I tend to believe that risk from non-AI sources is pretty low, and I’m therefore somewhat alarmed when I see people suggest or state relatively high probabilities of civilisational collapse without AI intervention. Could be worth trying to assess how widespread this view is and trying to argue directly against it.
I made this into two posts, my first LessWrong posts:
I think some kinds of content are uncontroversially good (e.g. posts that are largely informational rather than persuasive), so I think some people don’t have a trade-off here.
Yeah, agree this isn’t the usual argument from pro-progress people, who tend to be optimists who wouldn’t agree that current civilisation does more harm than good.
And yeah, my second bullet point is clearest when the end of factory farming is endogenous rather than exogenous. In the exogenous case it’s more confusing because I see progress pulling in both directions: in general, larger populations and more prosperity seem likely to increase economic demand for factory farms, and on the other hand, better alternatives in combination with social and political advocacy have the potential to decrease the number of animals used.
In the presence of exogenous ends, I suppose the track-skipping condition is precisely that the next track is worse than the track that (if you don’t skip) would have played but is stopped at the time of the exogenous event. In the specific case of factory farming, it’s not obvious to me what to expect in terms of which one is better.
It’s not really my specific area, but I had a quick look. (Frankly, this is mostly me just thinking out loud to see if I can come up with anything useful, and I don’t promise that I succeed.)
Yahoo Finance has option prices with expirations in Dec 2026. We’re mostly interested in upside potential rather than downside, so we look at call options, for which we see data up to strike prices of 280.[fn 1]
In principle I think the next step is to do something like invert Black-Scholes (perhaps (?) adjusting for the difference between European- and American-style options, assuming that these options are the latter), but that sounds hard, so let’s see if I can figure out something simpler from first principles:
The 280 strike Dec 2026 call option is the right to buy Nvidia stock, on Dec 18th 2026, for a price of $280. Nvidia’s current price is ~$124, so these options only have value if the stock more than doubles by then. They’re currently trading at $14.50, while the 275 call trades at $15.
The value of a particular option is the integral of the option’s payoff profile multiplied by the stock price’s probability density. If we want something like “probability the stock is at least X on date Y”, the ideal option payoff profile would be an indicator function with a step at X, but we can’t exactly get that. Instead, by buying a call struck at A and selling a call struck at B, we get a zero function up to A, then a linear increase from A to B, then a constant function from B. Picking A and B close together seems like the best approximation. It means looking at prices for very low-volume options, but looking at the nearby prices including for higher-volume options, they look superficially in line, so I’ll go with it.
More intuitively, if the stock was definitely going to be above both A and B, then the strike-A option would be B—A more valuable than the strike B option (that is, the right to buy a stock worth $10 for a price $1 is worth exactly $3 more than the right to do so for $4). If the stock was definitely going to be below both A and B, then both options would be worthless.
So the value of the two options differ by (B—A)P(the price is above B), plus some awkward term for when the price is between A and B, which you can hopefully make ignorable by making that interval small.
From this I hesitantly conclude that the options markets suggest that P(NVDA >= 280-ish) = 10%-ish?
[fn 1]: It looks like there are more strike prices than that, but all the ones after 280 I think aren’t applicable: you can see a huge discontinuity in the prices from 280 to 290, and all the “last trade date” fields for the higher options are from before mid-June, so I think these options don’t exist anymore and come from before the 10-to-1 stock split.
My impression is that GiveWell is on principle explicitly against treating different lives as having different values, even when the mechanism is as simple as different lives having different expected lengths (e.g. valuing saving someone from malaria less if you think they will have a lower life expectancy as a result of their illness).
From a straightforward, surface-level utilitarian perspective, this does seem like it would be a mistake. But I think there are good rule-utilitarian, systems-change, or co-operation-based reasons to want to take this stance.
Treating different lives as worth different amounts violates many people’s fairness and justice intuitions, which is both relevant from a moral-trade perspective (it’s a good co-operative principle to avoid doing things that other people really don’t want you to do) and a moral-uncertainty perspective (maybe they are correct to not want you to do those things!)
It’s also pretty uncomfortable that part of the reason that these people live in such difficult conditions is that previous generations have neglected to help them: a policy that discounts them on this basis will tend to reinforce existing problems.
As you say, the fact that they are the cheapest lives to save and the fact that they have poor state capacity are casually related, but they’re probably causally related the other way too, in the sense that it’s hard to demand or build good state capacity when you’re sick or barely feeding yourself. I hope that as material conditions improve in (say) the DRC, so too will political conditions; I’m no historian but that seems to have been the trend in much of the rest of the world.
(I have more to say but I have 2% phone battery so maybe I’ll make another comment later)
If I believe that:
current suffering of animals on factory farms outweighs the positive value of human lives,
eventually, technological and/or social progress will end factory farming
then I want to “skip the current track”.
This one might be for LW or the AF instead / as well, but I’d like to write a post about:
should we try to avoid some / all alignment research casually making it into the training sets for frontier AI models?
if so, what are the means that we can use to do this? how do they fare on the ratio between reduction in AI access vs. reduction in human access?
This became When “human-level” is the wrong threshold for AI
If this were the case, I would expect to see the most cost-effective cases saturated (leading to reduced marginal cost-effectiveness), but with tremendous improvement reducing child mortality globally. But that apparently hasn’t happened, when looking at the income for those charities.
Is it really justified to say that this hasn’t happened? After all, after you saturate all the most cost-effective interventions, all the previously-second-most cost-effective interventions become the most cost effective ones. There’s always something that is most cost effective and unfunded.
Child mortality has in fact dropped sharply in the last few decades, and I believe at least some of that is due to international aid work and health improvements. An example I hold in my head (that is years old now, so possibly stale) is that vaccine distribution is extremely cheap as a way to save lives, and the only reason it’s not an EA cause area is that it’s already funded by existing institutions, and the only remaining things to fund in vaccine access are the cases that are very difficult to reach, or more indirect things like improving take-up among populations that have access.
(To be clear I’m not saying this to let billionaires off the hook; I agree that in general many / most of them are failing their moral obligations. I’m not sure how much of the above work was funded by philanthropists rather than governments or grassroots movements. But I think it’s relevant context nonetheless.)
For publically-traded US companies there are ways to figure out the variance of their future value, not just the mean, mostly by looking at option prices. Unfortunately, OpenAI isn’t publically-traded and (afaik) has no liquid options market, but maybe other players (Nvidia? Microsoft?) can be more helpful there.
I don’t know if you’re even implying this, but the causal mechanism for altruism arising in humans doesn’t need to hold any moral force over us. Just because kin selection caused us to be altruistic, doesn’t mean we need to think “what would kin selection want?” when deciding how to be altruistic in future. We can replace the causal origin with our own moral foundations, and follow those instead.