I agree that it’s a difficult problem, but I’m not sure that it’s impossible.
Michael_Wiebe
Karma: 1,254
Yes, I think of EA as optimally allocating a budget to maximize social welfare, analogous to the constrained utility maximization problem in intermediate microeconomics.
The worldview diversification problem is in putting everything in common units (eg. comparing human and animal lives, or comparing current and future lives). Uncertainty over these ‘exchange rates’ translates into uncertainty in our optimal budget allocation.
Air pollution literature, relevant to OpenPhil.
literature on psychotherapy (vs cash transfers)
Yes, it sounds like MacAskill’s motivation is about PR and community health (“getting people out of bed in the morning”). I think it’s important to note when we’re funding things because of direct expected value, vs these indirect effects.
Does longtermism vs neartermism boil down to cases of tiny probabilities of x-risk?
When P(x-risk) is high, then both longtermists and neartermists max out their budgets on it. We have convergence.
When P(x-risk) is low, then the expected value is low for neartermists (since they only care about the next ~few generations) and high for longtermists (since they care about all future generations). Here, longtermists will focus on x-risks, while neartermists won’t.
Do we know the expected cost for training an AGI? Is that within a single company’s budget?
As you note, the key is being able to precisely select applicants based on altruism:
This tension also underpins a frequent argument made by policymakers that extrinsic rewards should be kept low so as to draw in agents who care sufficiently about delivering services per se. A simple conceptual framework makes precise that, in line with prevailing policy concerns, this attracts applicants who are less prosocial conditional on a given level of talent. However, since the outside option is increasing in talent, adding career benefits will draw in more talented individuals, and the marginal, most talented applicant in both groups will have the highest prosociality. Intuitively, since a candidate with high ability will also have a high outside option, if they are applying for the health worker position it must be because they are highly prosocial. The treatment effect on recruited candidates will therefore depend on how candidates are chosen from the pool. If applicants are drawn randomly, there might be a trade-off between talent and prosociality. However, if only the most talented are hired, there will be no trade-off.
Why does your graph have financial motivation as the y-axis? Isn’t financial motivation negatively correlated with altruism, by definition? In other words, financial motivation and altruism are opposite ends of a one-dimensional spectrum.
I would’ve put talent on the y-axis, to illustrate the tradeoff between talent and altruism.
So perhaps EA orgs can raise salaries and attract more-talented-yet-equally-commited workers. (Though this effect would depend on the level of the salary.)
Let be the computing power used to train the model. Is the idea that “if you could afford to train the model, then you can also afford for running models”?
Because that doesn’t seem obvious. What if you used 99% of your budget on training? Then you’d only be able to afford for running models.
Or is this just an example to show that training costs >> running costs?
Related:
”Losing Prosociality in the Quest for Talent? Sorting, Selection, and Productivity in the Delivery of Public Services”
By Nava Ashraf, Oriana Bandiera, Edward Davenport, and Scott S. LeeAbstract:
We embed a field experiment in a nationwide recruitment drive for a new health care position in Zambia to test whether career benefits attract talent at the expense of prosocial motivation. In line with common wisdom, offering career opportunities attracts less prosocial applicants. However, the trade-off exists only at low levels of talent; the marginal applicants in treatment are more talented and equally prosocial. These are hired, and perform better at every step of the causal chain: they provide more inputs, increase facility utilization, and improve health outcomes including a 25 percent decrease in child malnutrition.
https://ashrafnava.files.wordpress.com/2021/11/aer.20180326.pdf
Basically, is the computing power for training a fixed cost or a variable cost? If it’s a fixed cost, then there’s no further cost to using the same computing power to train models.
once the first human-level AI system is created, whoever created it could use the same computing power it took to create it in order to run several hundred million copies for about a year each.
How does computing power work here? Is it:
We use a supercomputer to train the AI, then the supercomputer is just sitting there, so we can use it to run models. Or:
We’re renting a server to do the training, and then have to rent more servers to run the models.
In (2), we might use up our whole budget on the training, and then not be able to afford to run any models.
Great comment. Perhaps it would be helpful to explicitly split the analysis by assumptions about takeoff speed? It seems that conditional on takeoff speed, there’s not much disagreement.
This paper makes that point about linear regressions in general.
Re: discount factor, longtermists have zero pure time preference. They still discount for exogenous extinction risk and diminishing marginal utility.
I’m very unsure how many people and how much funding the effective altruism community should be allocating to nuclear risk reduction or related research, and I think it’s plausible we should be spending either substantially more or substantially less labor and funding on this cause than we currently are (see also Aird & Aldred, 2022a).[6] And I have a similar level of uncertainty about what “intermediate goals”[7] and interventions to prioritize—or actively avoid—within the area of nuclear risk reduction (see Aird & Aldred, 2022b). This is despite me having spent approximately half my time from late 2020 to late 2021 on research intended to answer these questions, which is—unfortunately! - enough to make me probably among the 5-20 members of the EA community with the best-informed views on those questions. [bold added]
This is pretty surprising to me. Do you have a sense of how much uncertainty you could have resolved if you spent another half-year working on this?
Relevant, by @HaydnBelfield:
One possible response is about long vs short AI timelines, but that seems orthogonal to longtermism/neartermism.
Agreed, that’s another angle. NTs will only have a small difference between non-extinction-level catastrophes and extinction-level catastrophes (eg. a nuclear war where 1000 people survive vs one that kills everyone), whereas LTs will have a huge difference between NECs and ECs.