I am a Senior Programme Associate at Longview Philanthropy and a Research Affiliate at the Population Wellbeing Initiative at UT-Austin.
At Longview, I conduct grant investigations in artificial intelligence (AI) and long-term wellbeing. This includes ways to promote the safe development of AI and the exploration of new program areas to benefit future generations.
In addition, I continue to pursue research projects that I began during my Ph.D. in economics at Stanford University. I use a mix of applied, experimental, and historical methods to answer questions in public economics, political economy, and behavioral/experimental economics. My primary current research interest is the impacts of policies on future generations, and I have an additional interest in the economics of animal welfare. These interests have led me to work on long-term political dynamics and social preferences.
At Stanford, I was a National Science Foundation Graduate Research Fellow and a Global Priorities Fellow with the Forethought Foundation. I was previously a Senior Research Analyst at Northwestern University’s Global Poverty Research Lab and Innovations for Poverty Action. I am interested in effective altruism, including serving on the board of Animal Charity Evaluators.
zdgroff
Well I think MIC relies on some sort of discontinuity this century, and when we start getting into the range of precedented growth rates, the discontinuity looks less likely.
But we might not be disagreeing much here. It seems like a plausibly important update, but I’m not sure how large.
This is a valuable point, but I do think that giving real weight to a world where we have neither extinction nor 30% growth would still be an update to important views about superhuman AI. It seems like evidence against the Most Important Century thesis, for example.
It might be challenging to borrow (though I’m not sure), but there seem to be plenty of sophisticated entities that should be selling off their bonds and aren’t. The top-level comment does cut into the gains from shorting (as the OP concedes), but I think it’s right that there are borrowing-esque things to do.
I’m trying to make sure I understand: Is this (a more colorful version) of the same point as the OP makes at the end of “Bet on real rates rising”?
The other risk that could motivate not making this bet is the risk that the market – for some unspecified reason – never has a chance to correct, because (1) transformative AI ends up unaligned and (2) humanity’s conversion into paperclips occurs overnight. This would prevent the market from ever “waking up”.
However, to be clear, expecting this specific scenario requires both:
Buying into specific stories about how takeoff will occur: specifically, Yudkowskian foom-type scenarios with fast takeoff.
Having a lot of skepticism about the optimization forces pushing financial markets towards informational efficiency.
You should be sure that your beliefs are actually congruent with these requirements, if you want to refuse to bet that real rates will rise. Additionally, we will see that the second suggestion in this section (“impatient philanthropy”) is not affected by the possibility of foom scenarios.
It doesn’t seem all that relevant to me whether traders have a probability like that in their heads. Whether they have a low probability or are not thinking about it, they’re approximately leaving money on the table in a short-timelines world, which should be surprising. People have a large incentive to hunt for important probabilities they’re ignoring.
Of course, there are examples (cf. behavioral economics) of systemic biases in markets. But even within behavioral economics, it’s fairly commonly known that it’s hard to find ongoing, large-scale biases in financial markets.
Do you have a sense of whether the case is any stronger for specifically using cortical and pallial neurons? That’s the approach Romain Espinosa takes in this paper, which is among the best work in economics on animal welfare.
My husband and I are planning to donate to Wild Animal Initiative and Animal Charity Evaluators; we’ve also supported a number of political candidates this year (not tax deductible) who share our values.
We’ve been donating to WAI for a while, as we think they have a thoughtful, skilled team tackling a problem with a sweeping scale and scant attention.
We also support ACE’s work to evaluate and support effective ways to help animals. I’m on the board there, and we’re excited about ACE’s new approach to evaluations and trajectory for the coming years.
Yes, and thank you for the detailed private proposal you sent the research team. I didn’t see it but heard about it, and it seems like it was a huge help and just a massive amount of volunteer labor. I know they really appreciated it.
I’m an ACE board member, so full disclosure on that, though what I say here is in my personal capacity.
I’m very glad about a number of improvements to the eval process that are not obvious from this post. In particular, there are now numeric cost-effectiveness ratings that I found clarifying, overall explanations for each recommendation, and clearer delineation of the roles the “programs” and “cost-effectiveness” sections play in the reviews. I expect these changes to make recommendations more scope sensitive. This leaves me grateful for and confident in the new review framework.
As I noted on the nuclear post, I believe this is based on a (loosely speaking) person-affecting view (mentioned in Joel and Ben’s back-and-forth below). That seems likely to me to bias the cost-effectiveness downward.
Like Fin, I’m very surprised by how well this performs given takes in other places (e.g. The Precipice) on how asteroid prevention compares to other x-risk work.
Worth flagging that I believe this is based on a (loosely speaking) person-affecting view (mentioned in Joel and Ben’s back-and-forth below). That seems to me to bias the cost-effectiveness of anything that poses a sizable extinction risk dramatically downward.
At the same time, I find both the empirical work and the inside-view thinking here very impressive for a week’s work, and it seems like even those without a person-affecting view can learn a lot from this.
Thanks for writing this. I think about these sorts of things a lot. Given the title, do you know of examples of movements that did not start academic disciplines and appear to have suffered as a result?
The Global Priorities Institute and clusters of work around that do work in economics, including welfare economics. I’d also be curious to hear what you think they should do differently.
I’m toying with a project to gather reference classes for AGI-induced extinction and AGI takeover. If someone would like to collaborate, please get in touch.
(I’m aware of and giving thought to reference class tennis concerns but still think something like this is neglected.)
I don’t think it’s right that the broad project of alignment would look the same with and without considering religion. I’m curious what your reasoning is here and if I’m mistaken.
One way of reading this comment is that it’s a semantic disagreement about what alignment means. The OP seems to be talking about the problem of getting an AI to do the right thing, writ large, which may encompass a broader set of topics than alignment research as you define it.
Two other ways of reading it are that (a) solving the problem the OP is addressing (getting an AI to do the right thing, writ large) does not depend on values, or (b) solving the alignment problem will necessarily solve the value problem. I don’t entirely see how you can justify (a) without a claim like (b), though I’m curious if there’s a way.
You might justify (b) via the argument that solving alignment involves coming up a way to extrapolate values. Perhaps it is irrelevant which particular person you start with, because the extrapolation process will end up at the same point. To me this seems quite dubious. We have no such method and observe deep disagreement in the world. Which methods we use to resolve disagreement and determine whose values we include seem to involve a question of values. And from my lay sense, the methods of alignment that are currently most-discussed involve aligning it with specific preferences.
One thing that’s sad and perhaps not obvious to people is that, as I understand it, Nathan Robinson was initially sympathetic to EA (and this played a role in at-times vocal advocacy for animals). I don’t know that there’s much to be done about this. I think the course of events was perhaps inevitable, but that’s relevant context for other Forum readers who see this.
And worth noting that Ben Franklin was involved in the constitution, so at least some of his longtermist time seems to have been well spent.
I don’t have a strong view on the original setup, but I can clarify what the argument is. For the first point, that we maximize . The idea is that we want to maximize the likelihood that the organism chooses the action that leads to enjoyment (the one being selected for). That probability is a function of how much better it is to choose that action than the alternative. So if you get E from choosing that action and lose S from choosing the alternative, the benefit from choosing that action is E - (-S) = E + S. However, you only pay to produce the experience of the action you actually take. This last reason is why the costs are weighted by probability, while the benefits, which are only about the anticipation of the experience you would get conditional on your action, are not.
It occurs to me that a fuller model might endogenize n, i.e. be something like max P(E(C_E) + S(C_S)) s.t. P(.) C_E + (1 - P(.)) C_S = M. (Replacing n with 1 - P here so it’s a rate, not a level. Also, perhaps this reduces to the same thing based on the envelope theorem.)
And on the last point, that point is relevant for the interpretation of the model (e.g. choosing the value of n), but it is not an assumption of the model.
Like others, I really appreciate these thoughts, and it resonates with me quite a lot. At this point, I think the biggest potential failure mode for EA is too much drift in this direction. I think the “EA needs megaprojects” thing has generated a view that the more we spend, the better, which we need to temper. Given all the resources, there’s a good chance EA is around for a while and quite large and powerful. We need to make sure we put these tools to good use and retain the right values.
EA spending is often perceived as wasteful and self-serving
It’s interesting here how far this is from the original version of EA and its criticisms; e.g. that EA was an unrealistic standard that involved sacrificing one’s identity and sense of companionship for an ascetic universalism.
I think the old perception is likely still more common, but it’s probably a matter of time (which means there’s likely still time to change it). And I think you described the tensions brilliantly.
It seems like the critics would claim that EA is, if not coercing or subjugating, at least substantially influencing something like the world population in a way that meets the criteria for democratisation. This seems to be the claim in arguments about billionaire philanthropy, for example. I’m not defending or vouching for that claim, but I think whether we are in a sufficiently different situation may be contentious.