Are there any EAA researchers carefully tracking the potential of huge cost-effectiveness gains in the ag industry from genetic engineering advances of factory farmed animals? Or (less plausibly) advances from better knowledge/practice/lore from classical artificial selection? As someone pretty far away from the field, a priori the massive gains made in biology/genetics in the last few decades seems like something that we plausibly have not priced in in. So it’d be sad if EAAs get blindsided by animal meat becoming a lot cheaper in the next few decades (if this is indeed viable, which it may not be).
I wrote this last Summer as a private “blog post” just for me. I’m posting it publicly now (after mild editing) because I have some vague idea that it can be good to make things like this public. These rambling thoughts come from my very naive point of view (as it was in the Summer of 2020; not to suggest my present day point of view is much less naive). In particular if you’ve already read lots of moral philosophy you probably won’t learn anything from reading this.
The free-for-all
Generally, reading various moral philosophy writings has probably made me (even) more comfortable trusting my own intuitions / reasoning regarding what “morality” is and what the “correct moral theory” is.
I think that, when you start engaging with moral philosophy, there’s a bit of a feeling that when you’re trying to reason about things like what’s right and wrong, which moral theory is superior to the other, etc, there are some concrete rules you need to follow, and (relatedly) certain words or phrases have a solid, technical definition that everyone with sufficient knowledge knows and agrees on. The “certain words or phrases” I have in mind here are things like “morally right”, “blameworthy”, “ought”, “value”, “acted wrongly”, etc.
To me right now, the situation seems a bit more like the following: moral philosophers (including knowledgeable amateurs, etc) have in mind definitions for certain words, but these definitions may be more or less precise, might change over time, and differ from person to person. And in making a “moral philosophy” argument (say, writing down an argument for a certain moral theory), the philosopher can use flexibility of interpretation as a tool to make their argument appear more forceful than it really is. Or, the philosopher’s argument might imply that certain things are self-evidently true, and the reader might be (maybe unconsciously) fooled into thinking that this is the case, when in fact it isn’t.
It seems to me now that genuinely self-evident truths are in very short supply in moral philosophy. And, now that I think this is the case, I feel like I have much more licence to make up my own mind about things. That feels quite liberating.
But it does also feel potentially dangerous. Of course, I don’t think it’s dangerous that *I* have freedom to decide what “doing good” means to me. But I might find it dangerous that others have that freedom. People can consider committing genocide to be “doing what is right” and it would be nice to have a stronger argument against this than “this conflicts with my personal definition of what good is”. And, of course, others might well think it’s dangerous that I have the freedom to decide what doing good means.
What does morality even mean?
Now that we’re in this free-for-all, even defining morality seems problematic.
I suppose I can make some observations about myself, like
When I see injustice in the world, I feel a strong urge to do something about it
When I see others suffering, I want to relieve that suffering
I have a strong intuition that it’s conscious experience that ultimately matters—“what you don’t know can’t hurt you” is, I think, literally true
And some conscious experiences are clearly very bad (and some are clearly very good)
And so on
I guess these things are all in the region of “wanting to improve the lives of others”. This sounds a lot like wanting to do what is morally good / morally praiseworthy, and seems at least closely related to morality.
In some ways, whether I label some of my goals and beliefs as being to do with “morality” doesn’t matter—either way, it seems clear that the academic field of moral philosophy is pretty relevant. And presumably when people talk about morality outside of an academic context, they’re at least sometimes talking about roughly the thing I’m thinking of.
A thought on how we describe existential risks from misaligned AI:
Sometimes discussions focus on a fairly specific version of AI risk, which involves humanity being quickly wiped out. Increasingly, though, the emphasis seems to be on the more abstract idea of “humanity losing control of its future.” I think it might be worthwhile to unpack this latter idea a bit more.
There’s already a fairly strong sense in which humanity has never controlled its own future. For example, looking back ten thousand years, no one decided that the sedentary agriculture would increasingly supplant hunting and gathering, that increasingly complex states would arise, that slavery would become common, that disease would take off, that social hierarchies and gender divisions would become stricter, etc. The transition to the modern world, and everything that came with this transition, also doesn’t seem to have been meaningfully chosen (or even really understood by anyone). The most serious effort to describe a possible future in detail — Hanson’s Age of Em — also describes a future with loads of features that most present-day people would not endorse.
As long as there are still strong competitive pressures or substantial random drift, it seems to me, no generation ever really gets to choose the future.[1] It’s actually sort of ambiguous, then, what it means to worry about “losing control of our future.”
Here are a few alternative versions of the concern that feel a bit crisper to me:
If we ‘mess up on AI,’ then even the most powerful individual humans will have unusually little influence over their own lives or the world around them.[2]
If we ‘mess up on AI,’ then future people may be unusually dissatisfied about the world they live in. In other words, people’s preferences will be unfilled to an unusually large degree.
Humanity may have a rare opportunity to take control of its own future, by achieving strong coordination and then locking various things in. But if we ‘mess up on AI,’ then we’ll miss out on this opportunity.[3]
Something that’s a bit interesting about these alternative versions of the concern, though, is that they’re not inherently linked to AI alignment issues. Even if AI systems behave roughly as their users intend, I believe each of these outcomes is still conceivable. For example, if there’s a missed opportunity to achieve strong coordination around AI, the story might look like the failure of the Baruch Plan for international control of nuclear weapons: that failure had much more to do with politics than it had to do with the way engineers designed the technology in question.
In general, if we move beyond discussing very sharp alignment-related catastrophes (e.g. humanity being quickly wiped out), then I think concerns about misaligned AI start to bleed into broader AI governance concerns. It starts to become more ambiguous whether technical alignment issues are actually central or necessary to the disaster stories people tell.
Although, admittedly, notable individuals or groups (e.g. early Christians) do sometimes have a fairly lasting and important influence. ↩︎
As an analogy, in the world of The Matrix, people may not actually have much less control over the long-run future than hunter-gatherers did twenty thousand years ago. But they certainly have much less control over their own lives. ↩︎
Notably, this is only a bad thing if we expect the relevant generation of humans to choose a better future than would be arrived at by default. ↩︎
Do you have the intuition that absent further technological development, human values would drift arbitrarily far? It’s not clear to me that they would—in that sense, I do feel like we’re “losing control” in that even non-extinction AI is enabling a new set of possibilities that modern-day humans would endorse much less than the decisions of future humans otherwise. (It does also feel like we’re missing the opportunity to “take control” and enable a new set of possibilities that we would endorse much more.)
Relatedly, it doesn’t feel to me like the values of humans 150,000 years ago and humans now and even ems in Age of Em are all that different on some more absolute scale.
Do you have the intuition that absent further technological development, human values would drift arbitrarily far?
Certainly not arbitrarily far. I also think that technological development (esp. the emergence of agriculture and modern industry) has played a much larger role in changing the world over time than random value drift has.
[E]ven non-extinction AI is enabling a new set of possibilities that modern-day humans would endorse much less than the decisions of future humans otherwise.
I definitely think that’s true. But I also think that was true of agriculture, relative to the values of hunter-gatherer societies.
To be clear, I’m not downplaying the likelihood or potential importance of any of the three crisper concerns I listed. For example, I think that AI progress could conceivably lead to a future that is super alienating and bad.
I’m just (a) somewhat pedantically arguing that we shouldn’t frame the concerns as being about a “loss of control over the future” and (b) suggesting that you can rationally have all these same concerns even if you come to believe that technical alignment issues aren’t actually a big deal.
FWIW, I wouldn’t say I agree with the main thesis of that post.
However, while I expect machines that outcompete humans for jobs, I don’t see how that greatly increases the problem of value drift. Human cultural plasticity already ensures that humans are capable of expressing a very wide range of values. I see no obviously limits there. Genetic engineering will allow more changes to humans. Ems inherit human plasticity, and may add even more via direct brain modifications.
In principle, non-em-based artificial intelligence is capable of expressing the entire space of possible values. But in practice, in the shorter run, such AIs will take on social roles near humans, and roles that humans once occupied....
I don’t see why people concerned with value drift should be especially focused on AI. Yes, AI may accompany faster change, and faster change can make value drift worse for people with intermediate discount rates. (Though it seems to me that altruistic discount rates should scale with actual rates of change, not with arbitrary external clocks.)
I definitely think that human biology creates at least very strong biases toward certain values (if not hard constraints) and that AI system would not need to have these same biases. If you’re worried about future agents having super different and bad values, then AI is a natural focal point for your worry.
A couple other possible clarifications about my views here:
I think that the outcome of the AI Revolution could be much worse, relative to our current values, than the Neolithic Revolution was relative to the values of our hunter-gatherer ancestors. But I think the question “Will the outcome be worse?” is distinct from the question “Will we have less freedom to choose the outcome?”
I’m personally not so focused on value drift as a driver of long-run social change. For example, the changes associated with the Neolithic Revolution weren’t really driven by people becoming less egalitarian, more pro-slavery, more inclined to hold certain religious beliefs, more ideologically attached to sedentism/farming, more happy to accept risks from disease, etc. There were value changes, but, to some significant degree, they seem to have been downstream of technological/economic change.
Really appreciate the clarifications! I think I was interpreting “humanity loses control of the future” in a weirdly temporally narrow sense that makes it all about outcomes, i.e. where “humanity” refers to present-day humans, rather than humans at any given time period. I totally agree that future humans may have less freedom to choose the outcome in a way that’s not a consequence of alignment issues.
I also agree value drift hasn’t historically driven long-run social change, though I kind of do think it will going forward, as humanity has more power to shape its environment at will.
I also agree value drift hasn’t historically driven long-run social change
My impression is that the differences in historical vegetarianism rates between India and China, and especially India and southern China (where there is greater similarity of climate and crops used), is a moderate counterpoint. At the timescale of centuries, vegetarianism rates in India are much higher than rates in China. Since factory farming is plausibly one of the larger sources of human-caused suffering today, the differences aren’t exactly a rounding error.
I do agree that quasi-random variation in culture can be really important. And I agree that this variation is sometimes pretty sticky (e.g. Europe being predominantly Christian and the Middle East being predominantly Muslim for more than a thousand years). I wouldn’t say that this kind of variation is a “rounding error.”
Over sufficiently long timespans, though, I think that technological/economic change has been more significant.
As an attempt to operationalize this claim: The average human society in 1000AD was obviously very different than the average human society in 10,000BC. I think that the difference would have been less than half as large (at least in intuitive terms) if there hadn’t been technological/economic change.
I think that the pool of available technology creates biases in the sorts of societies that emerge and stick around. For large enough amounts of technological change, and long enough timespans (long enough for selection pressures to really matter), I think that shifts in these technological biases will explain a large portion of the shifts we see in the traits of the average society.[1]
If selection pressures become a lot weaker in the future, though, then random drift might become more important in relative terms. ↩︎
Another interpretation of the concern, though related to your (3), is that misaligned AI may cause humanity to lose the potential to control its future. This is consistent with humanity not having (and never having had) actual control of its future; it only requires that this potential exists, and that misaligned AI poses a threat to it.
Something that came up with a discussion with a coworker recently is that often internet writers want some (thoughtful) comments, but not too many, since too many comments can be overwhelming. Or at the very least, the marginal value of additional comments is usually lower for authors when there are more comments.
However, the incentives for commentators is very different: by default people want to comment on the most exciting/cool/wrong thing, so internet posts can easily by default either attract many comments or none. (I think) very little self-policing is done, if anything a post with many comments make it more attractive to generate secondary or tertiary comments, rather than less.
Meanwhile, internet writers who do great work often do not get the desired feedback. As evidence: For ~ a month, I was the only person who commented on What Helped the Voiceless? Historical Case Studies (which later won the EA Forum Prize).
This will be less of a problem if internet communication is primarily about idle speculations and cat pictures. But of course this is not the primary way I and many others on the Forum engage with the internet. Frequently, the primary publication venue for some subset of EA research and thought is this form of semi-informal internet communication.
Academia gets around this problem by having mandated peer review for work, with a minimum number of peer reviewers who are obligated to read (or at least skim) anything that passes through an initial filter.
It is unclear how (or perhaps if?) we ought to replicate something similar in a voluntary internet community.
On a personal level, the main consequence of this realization is that I intend to become slightly more inclined to comment more on posts without many prior comments, and correspondingly less to posts with many comments (especially if the karma:comment ratio is low).
On a group level, I’m unsure what behavior we ought to incentivize, and/or how to get there. Perhaps this is an open question that others can answer?
I think these are useful observations and questions. (Though I think “too many comments” should probably be much less of a worry than “too few”, at least if the comments make some effort to be polite and relevant, and except inasmuch as loads of comments on one thing sucks up time that could be spent commenting on other things where that’d be more useful.)
I think a few simple steps that could be taken by writers are:
People could more often send google doc drafts to a handful of people specifically selected for being more likely than average to (a) be interested in reading the draft and (b) have useful things to say about it
People could more often share google doc drafts in the Effective Altruism Editing & Review Facebook group
People could more often share google doc drafts in other Facebook groups, Slack workspaces, or the like
E.g., sharing a draft relevant to improving institutional decision-making in the corresponding Facebook group
People could more often make posts/shortforms that include an executive summary (or similar) and a link to the full google doc draft, saying that this is still like a draft and they’d appreciate comment
Roughly this has been done recently by Joe Carlsmith and Ben Garfinkel, for example
This could encourage more comments that just posting the whole thing to the Forum as a regular post, since (a) this conveys that this is still a work-in-progress and that comments are welcome, and (b) google docs make it easier to comment on specific points
When people do post full versions of things on the Forum (or wherever), they could explicitly indicate that they’re interested in feedback, indicate roughly what kinds of feedback would be most valuable, and indicate that they might update the post in light of feedback (if that’s true)
People could implement the advice given in these two good posts:
I think a few simple steps that could be taken by potential reviewers are:
As you suggest, people could adjust their behaviours a bit more towards commenting on posts without many prior comments, and save time to do that by commenting a bit less on posts with many comments
Likewise, people could adjust their behaviours a bit more towards commenting on drafts they come across where the author is seeking feedback (e.g., drafts that were sent to the person directly or shared in some group the person is part of), especially if they don’t yet have many prior comments
Sometimes people worry that a post idea might be missing some obvious, core insight, or just replicating some other writing you haven’t come across. I think this is mainly a problem only inasmuch as it could’ve been more efficient for you to learn things than slowly craft a post.
So if you can write (a rough version of) the post quickly, you could just do that.
Or you could ask around or make a quick Question post to outline the basic idea and ask if anyone knows of relevant things you should read.
I’m considering donating to the Centre for Women’s Justice. With a budget of about £300k last year, they have undertaken strategic litigation against the government , Crown prosecutors etc for mismanagement of sexual assault cases. The cases seem well-chosen to raise the issue on the political agenda. I think more rapists being successfully prosecuted would have a very positive impact so I’m excited to see this work. I’m planning to email them soon.
https://www.centreforwomensjustice.org.uk/strategic-plan
Some initial thoughts on “Are We Living At The Hinge Of History”?
In the below I give a very rough summary of Will MacAskill’s article Are We Living At The Hinge Of History? and give some very preliminary thoughts on the article and some of the questions it raises.
I definitely don’t think that what I’m writing here is particularly original or insightful: I’ve thought about this for no more than a few days, any points I make are probably repeating points other people have already made somewhere, and/or are misguided, etc. This seems like an incredibly deep topic which I feel like I’ve barely scratched the surface of. Also, this is not a focussed piece of writing trying to make a particular point, it’s just a collection of thoughts on a certain topic.
(If you want to just see what I think, skip down to “Some thoughts on the issues discussed in the article”)
The Hinge of History claim (HH): we are among the most influential people ever (past or future). Influentialness is, roughly, how much good a particular person at a particular time can do through direct expenditure of resources (rather than investment)
Two prominent longtermist EA views imply HH
Two worldviews prominent in longtermist EA imply that HH is true:
Time of Perils view: we live at a time of unusually high extinction risk, and we can do an unusual amount to reduce this risk
Value Lock-In view: we’ll soon invent a technology that allows present-day agents to assert their values indefinitely into the future (in the Bostrom-Yudkowsky version of this view, the technology is AI)
Arguments against HH
The base rates argument
Claim: our prior should be that we’re as likely as anyone else, past or present, to be the most influential person ever (Bostrom’s Self-Sampling Assumption (SSA)). Under this prior, it’s astronomically unlikely that any particular person is the most influential person ever.
Then the question is how much should we update from this prior
The standard of evidence (Bayes factor) required to favour HH is incredibly high. E.g. we need a Bayes factor of ~107 to move from a 1 in 100 million credence to a 1 in 10 credence. For comparison, a p=0.05 result from a randomised controlled trial gives a Bayes factor of 3 under certain reasonable assumptions.
The arguments for Time of Perils or Value Lock-In might be somewhat convincing; but hard to see how they could be convincing enough
E.g. our track record of understanding the importance of historical events is very poor
When considering how much to update from the prior, we should be aware that there are biases that will tend to make us think HH is more likely than it really is
Counterargument 1: we only need to be at an enormously influential time, not the most influential, and the implications are ~the same either way
Counter 1 to counterargument 1: the Bostrom-Yudkowsky view says we’re at the most influential time ever, so you should reject the Bostrom-Yudkowsky view if you’re abandoning the idea that we’re at the most influential time ever. So there is a material difference between “enormously influential time” and “most influential time”.
Counter 2 to counterargument 1: if we’re not at the most influential time, presumably we should transfer our resources forward to the most influential time, so the difference between “enormously influential time” and “most influential time” is highly action-relevant.
Counterargument 2: the action-relevant thing is the influentialness of now compared to any time we can pass resources on to
Again the Bostrom-Yudkowsky view is in conflict with this
But MacAskill concedes that it does seem right that this is the action-relevant thing. So e.g. we could assume we can only transfer resources 1000 years into the future and define Restricted-HH: we are among the most influential people out of the people who will live over the next 1000 years
The inductive argument
Claim: The influentialness of comparable people has been increasing over time, and we should expect this to continue, so the influentialness of future people who we can pass resources onto will be greater
Evidence: if we consider the state of knowledge and ethics in 1600 vs today, or in 1920/1970 vs today, it seems clear that we have more knowledge and better ethics now than we did in 1600 or in 1920/1970
And seems clear that there are huge gaps in our knowledge today (so doesn’t seem that we should expect this trend to break)
Arguments for HH
Single planet
Argument 1: we’re living on a single planet, implying greater influentialness
Implies particular vulnerabilities e.g. asteroid strikes
Implies individual people have an unusually large fraction of total resources
Implies instant global communication
Counterargument:
Asteroids are not a big risk
For other prominent risks like AI or totalitarianism, being on multiple planets doesn’t seem to help
We might well have quite a long future period on earth (1000s or 10,000s of years), which makes being on earth now less special
And in the early stages of space settlement the picture isn’t necessarily that relevantly different to the single planet one
Rapid growth
Argument 2: we’re now in a period of unusually fast economic and tech progress, implying greater influentialness. We can’t maintain the present-day growth rate indefinitely.
MacAskill seems sympathetic to the argument, but says it implies not that today is the most important time, but that the most important time is some time might be in the next few thousand years
Also, maybe longtermist altruists are less influential during periods of fast economic growth because rapid change makes it harder to plan reliably
And comparing economic power across long timescales is difficult
Other arguments
A few other arguments for HH are briefly touched on in a footnote: that existential risk / value lock-in lowers the number of future people in the reference class for the influentialness prior; that we might choose other priors that are more favourable for HH, and that earlier people can causally affect more future people
Some quick meta-level thoughts on the article
I wish it had a detailed discussion about choosing a prior for influentialness, which I think is really important.
There’s a comment that the article ignores the fact that the annual risk of extinction or lock-in in the future has implications for present-day influentialness because in Trammell’s model this is incorporated into the pure rate of time preference. I find that pretty weird. Trammell’s model is barely referenced elsewhere in the paper so I don’t really see why we should neglect to discuss something just because it happens to be interpreted in a certain way within his model. Maybe I missed the point here.
I think it’s a shame that MacAskill doesn’t really give numbers in the article for his prior and posterior, either on HH or restricted-HH (this EA Forum comment thread by Greg Lewis is relevant).
Some thoughts on the issues discussed in the article
Two main points from the article
It kind of feels like there are two somewhat independent things that are most interesting from the article:
1. The claim: we should reject the Time of Perils view, and the Bostrom-Yudkowsky view, because in both cases the implication for our current influentialness is implausible
2. The question: what do high level / relatively abstract arguments tell us about whether we can do the most good by expending resources now or by passing resources on to future generations?
Avoiding rejecting the Time of Perils and Bostrom-Yudkowsky views
I think there are a few ways we can go to avoid rejecting the Time of Perils and Bostrom-Yudkowsky views
We can find the evidence in favour of them strong enough to overwhelm the SSA prior through conventional Bayesian updating
We can find the evidence in favour of them weaker than in the previous case, but still strong enough that we end up giving them significant credence in the face of the SSA prior, through some more forgiving method than Bayesian updating
We can use a different prior, or claim that we should be uncertain between different priors
Or we can just turn the argument (back?) around, and say that the SSA prior is implausible because it implies such a low probability for the Time of Perils and Bostrom-Yudkowsky views. Toby Ord seems to say something like this in the comments to the EA Forum post (see point 3).
A nearby alternative is to modify the Time of Perils and Bostrom-Yudkowsky views a bit so that they don’t imply we’re among the most influential people ever. E.g. for the Bostrom-Yudkowsky view we could make the value lock-in a bit “softer” by saying that for some reason, not necessarily known/stated, the lock-in would probably end after some moderate (on cosmological scales) length of time. I’d guess that many people might find a modified view more plausible even independently of the influentialness implications.
I’m not really sure what I think here, but I feel pretty sympathetic to the idea that we should be uncertain about the prior and that this maybe lends itself to having not too strong a prior against the Time of Perils and Bostrom-Yudkowsky views.
On the question of whether to expend resources now or later
The arguments MacAskill discusses suggest that the relevant time frame is the next few thousand years (because the next few thousand years seem (in expectation) especially high influentialness and because it might be effectively impossible to pass our resources further into the future).
It seems like the pivotal importance of priors on influentialness (or similar) then evaporates: it no longer seems that implausible on the SSA prior that now is a good time to expend resources rather than save. E.g. say there’ll be a 20 year period in the next 1000 years where we want to expend philanthropic resources rather than save them to pass on to future generations. Then a reasonable prior might be that we have a 20/1000 = 1 in 50 chance of being in that period. That’s a useful reference point and is enough to make us skeptical about arguments that we are in such a period, but it doesn’t seem overwhelming. In fact, we’d probably want to spend at least some resources now even purely based on this prior.
In particular, it seems like some kind of detailed analysis is needed, maybe along the lines of Trammell’s model or at least using that model as a starting point. I think many of the arguments in MacAskill’s article should be part of that detailed analysis, but, to stress the point, they don’t seem decisive to me.
In the article, the Inductive Argument is supported by the idea of moral progress: MacAskill cites the apparent progress in our moral values over the past 400 years as evidence for the idea that we should expect future generations to have better moral values than we do. Obviously, whether we should expect moral progress in the future is a really complex question, but I’m at least sympathetic to the idea that there isn’t really moral progress, just moral fashions (so societies closer in time to ours seem to have better moral values just because they tend to think more like us).
Of course, if we don’t expect moral progress, maybe it’s not so surprising that we have very high influentialness: if past and future actors don’t share our values, it seems very plausible on the face of it that we’re better off expending our resources now than passing them off to future generations in the hope they’ll carry out our wishes. So maybe MacAskill’s argument about influentialness should update us away from the idea of moral progress?
But if we’re steadfast in our belief in moral progress, maybe it’s not so surprising that we have high influentialness because we find ourselves in a world where we are among the very few with a longtermist worldview, which won’t be the case in the future as longtermism becomes a more popular view. (I think Carl Shulman might say something like this in the comments to the original EA Forum post)
My overall take
I think “how plausible is this stuff under an SSA prior” is a useful perspective
Still, thinking about this hasn’t caused me to completely dismiss the Time of Perils View or the Bostrom-Yudkowsky view (I probably already had some kind of strong implausibility prior on those views).
The arguments in the article are useful for thinking about how much (e.g.) the EA longtermist community should be spending rather than saving now, but a much more detailed analysis seems necessary to come to a firm view on this.
A quote to finish
I like the way the article ends, providing some motivation for the Inductive Argument in a way I find appealing on a gut level:
Just as our powers to grow crops, to transmit information, to discover the laws of nature, and to explore the cosmos have all increased over time, so will our power to make the world better — our influentialness. And given how much there is still to understand, we should believe, and hope, that our descendents look back at us as we look back at those in the medieval era, marvelling at how we could have got it all so wrong.
Some initial thoughts on moral realism vs anti-realism
I wrote this last Summer as a private “blog post” just for me. I’m posting it publicly now (after mild editing) because I have some vague idea that it can be good to make things like this public. These thoughts come from my very naive point of view (as it was in the Summer of 2020; not to suggest my present day point of view is much less naive). In particular if you’ve already read lots of moral philosophy you probably won’t learn anything from reading this. Also, I hope my summaries of other people’s arguments aren’t too inaccurate.
Recently, I’ve been trying to think seriously about what it means to do good. A key part of Effective Altruism is asking ourselves how we can do the most good. Often, considering this question seems to be mostly an empirical task: how many lives will be saved through intervention A, and how many through intervention B? Aside from the empirical questions, though, there are also theoretical ones. One key consideration is what we mean by doing good.
Moral Philosophy
There is a branch of philosophy called moral philosophy which is (partly) concerned with answering this question.
It’s important to me that I don’t get too drawn into the particular framings that have evolved within the academic discipline of moral philosophy, which are, presumably, partly due to cultural or historical forces, etc. This is because I really want to try to come up with my own view, and I think that (for me) the best process for this involves not taking other people’s views or existing works too seriously, especially while I try to think about these things seriously for the first time.
Still, it seems useful to get familiar with the major insights and general way of thinking within moral philosophy, because
I’ll surely learn a lot of useful stuff
I’ll be able to communicate with other people who are familiar with moral philosophy (which probably includes most of the most interesting people to talk to on this topic).
Moral realism
I’ve read a couple of Stanford Encyclopedia of Philosophy articles, and a series of posts by Lukas Gloor arguing for moral anti-realism.
I found the Stanford Encyclopedia of Philosophy articles fairly tough going but still kind of useful. I thought the Gloor posts were great.
The Gloor posts have kind of convinced me to take the moral anti-realist side, which, roughly, denies the existence of definitive moral truths.
While I suppose I might consider my “inside view” to be moral anti-realist at the moment, I can easily see this changing in the future. For example, I imagine that if I read a well-argued case for moral realism, I might well change my mind.
In fact, prior to reading Gloor’s posts, I probably considered myself to be a moral realist. I think I’d heard arguments, maybe from Will MacAskill, along the lines that i) if moral anti-realism is true, then nothing matters, whereas if realism is true, you should do what the true theory requires you to do, and ii) there’s some chance that realism is true, therefore iii) you should do what the true theory requires you to do.
Gloor discusses an argument like this in one of his posts. He calls belief in moral realism founded on this sort of argument “metaethical fanaticism” (if I’ve understood him correctly).
I’m not sure that I completely understood everything in Gloor’s posts. But the “fanaticism” label does feel appropriate to me. It feels like there’s a close analogy with the kinds of fanaticism that utilitarianism is susceptible to, for example. An example of that might be a Pascal’s wager type argument—if there’s a finite probability that I’ll get infinite utility derived from an eternal life in a Christian heaven, I should do what I can to maximise that probability.
It feels like something has gone wrong here (although admittedly it’s not clear what), and this Pascal’s wager argument doesn’t feel at all like a strong argument for acting as if there’s a Christian heaven. Likewise, the “moral realist wager” doesn’t feel like a strong argument for acting as if moral realism is true, in my current view.
Moral anti-realism
Gloor also argues that we don’t lose anything worth having by being moral anti-realists, at least if you’re his brand of moral anti-realist. I think he calls the view he favours “pluralistic moral reductionism”.
On his view, you take any moral view (or maybe combination of views) you like. These can (and maybe for some people, “should”) be grounded in our moral intuitions, and maybe use notions of simplicity of structure etc, just as a moral realist might ground their guess(?) at the true moral theory in similar principles. Your moral view is then your own “personal philosophy”, which you choose to live by.
One unfortunate consequence of this view is that you don’t really have any grounds to argue with someone else who happens to have a different view. Their view is only “wrong” in the sense that it doesn’t agree with yours; there’s no objective truth here.
From this perspective, it would arguably be nicer if everyone believed that there was a true moral view that we should strive to follow (even if we don’t know what it is). Especially if you also believe that we could make progress towards that true moral view.
I’m not sure how big this effect is, but it feels like more than nothing. So maybe I don’t quite agree that we don’t lose anything worth having by being moral anti-realists.
In any case, the fact that we might wish that moral realism is true doesn’t (presumably) have any bearing on whether or not it is true.
Reflections
I already mentioned that reading Gloor’s posts has caused me to favour moral anti-realism. Another effect, I think, is that I am more agnostic about the correct moral theory. Some form of utilitarianism, or at least consequentialism, seems far more plausible to me as the moral realist “one true theory” than a deontological theory or virtue ethics theory. Whereas if moral anti-realism is correct, I might be more open to non-consequentialist theories. (I’m not sure whether this new belief would stand up to a decent period of reflection, though—maybe I’d be just as much of a convinced moral anti-realist consequentialist after some reflection).
The longer you hold a view, and the more publicly you hold a view, the more calcified it typically becomes. Changing your mind becomes more aversive and potentially costly, you have more tools at your disposal to mount a lawyerly defense, and you find it harder to adopt frameworks/perspectives other than your favored one (the grooves become firmly imprinted into your brain). At least, this is the way it seems and personally feels to me.[1]
For this reason, the observation “someone I respect publicly argued for X many years ago and still believes X” typically only provides a bit more evidence than the observation “someone I respect argued for X many years ago.” For example, even though I greatly respect Daron Acemoglu, I think the observation “Daron Acemoglu still believes that political institutions are the central determinant of economic growth rates” only gives me a bit more evidence than the observation “15 years ago Daron Acemoglu publicly argued that institutions are the central determinant of economic growth rates.”
A corollary: If there’s an academic field that contains a long-standing debate, and you’d like to defer to experts in this field, you may want to give disproportionate weight to the opinions of junior academics. They’re less likely to have responded to recent evidence and arguments in an epistemically inflexible way.
Of course, there are exceptions. The final chapter of Scout Mindset includes a moving example of a professor publicly abandoning a view he had championed for fifteen years, after a visiting academic presented persuasive new evidence. The reason these kinds of stories are moving, though, is that they describe truly exceptional behavior. ↩︎
At least in software, there’s a problem I see where young engineers are often overly bought-in to hype trains, but older engineers (on average) stick with technologies they know too much.
I would imagine something similar in academia, where hot new theories are over-valued by the young, but older academics have the problem you describe.
That consideration—and the more basic consideration that more junior people often just know less—definitely pushes in the opposite direction. If you wanted to try some version of seniority-weighted epistemic deference, my guess is that the most reliable cohort would have studied a given topic for at least a few years but less than a couple decades.
There are other formats that may make sense, like tags for material on this forum, or wikis. But the general principle is that you can do something really useful by making it easy for people to find existing material on a topic.
I mean, it’s pretty relevant for peace (I guess most wars result from conflict of factions or succession crises) and for a well functioning government. People talk about the dangers of polarization, about why nations fail, or autoritarianism, or iidm… It’s not neglected per se (it’s been the focus of some of classical works in political phil & sci), but I’m not sure all low-hanging has been eaten; plus, thinking about interventions as increasing / decreasing political stability might help assessing other areas (like IIDM).
The article was far better than I expect most reporting on climate change as a potential existential risk to be
This is in line with Kelsey Piper generally seeming to do great work
I particularly appreciated that it (a) emphasised how the concepts of catastrophes in general and extinction in particular are distinct and why that matters, but (b) did this in a way that I suspect has a relatively low risk of seeming callous, nit-picky, or otherwise annoying to people who care about climate change
But I also had some substantive issues with the article, which I’ll discuss below
The article conflated “existential threat”/“existential risk” with “extinction risk”, thereby ignoring two other types of existential catastrophe: unrecoverable collapse and unrecoverable dystopia
Some quotes from the article to demonstrate what the conflation I’m referring to:
“But there’s a standard meaning of that phrase [existential threat]: that it’s going to wipe out humanity — or even, as Warren implied Wednesday night, all life on our planet.”
“To academics in philosophy and public policy who study the future of humankind, an existential risk is a very specific thing: a disaster that destroys all future human potential and ensures that no generations of humans will ever leave Earth and explore our universe.”
I think this also means the article kind-of ignores or overconfidently dismisses the possibility that climate change might cause unrecoverable collapse or unrecoverable dystopia
(The article does mention collapse a few times, but not something that corresponds to unrecoverable collapse or unrecoverable dystopia)
I do think it is very unlikely (e.g., below 1 in 1000 chance) that climate change would relatively directly cause those things, but that’s only a tentative view, and I think uncertainty and further research is warranted
I expect that the fact that article kind-of ignores unrecoverable collapse would cause some non-EA-types to disagree with the article, and that they’d be right to do so
(Though I also expect that many of these people would be overconfident that climate change would cause a major collapse, that they’d pay insufficient attention to the question of whether this collapse is “unrecoverable”, and that they also wouldn’t consider reasonable scenarios of dystopias)
The article also failed to mention the term “existential risk factor” or allude to that basic idea
It thus ignores or overconfidently dismisses the possibility that climate change might make x-risks more likely, even if it doesn’t directly cause them
Finally, the article failed to mention one other, important reason why the distinction between catastrophes in general and extinction in particular matters: This distinction is relevnat to prioritisation, in a world where other things may indeed be plausible extinction risks
If people care about climate change because they think it’s fairly likely to cause extinction, but in reality it’s less likely to cause extinction and something else is more likely to do so, then (in theory!), by their own values, it should be really important for them to learn that and adjust their priorities
But this is more just something I think could’ve maybe made the article even better, rather than something that I felt was problematic about it
I think it’s totally understandable that Kelsey Piper didn’t discuss the above points in detail, but I think she could’ve:
Been more careful in her phrasing to at least avoid being actively misleading
Maybe briefly touched on (some) of these points and provided links to where they’re discussed more thoroughly
(Disclaimer-ish thing: I haven’t sent this to Kelsey because it doesn’t seem super important, the article is from 2019, I assume she’s quite busy, and I’m posting this as a shortform rather than something super prominent.)
Small request for help to measure the impact of an intervention we have planned:
In short, Animal Rebellion UK is planning a large protest/civil resistance action and I’m keen to try quantitatively and semi-rigorously measure the impact of this, budget permitting. We’re planning on doing an opinion poll (or another bit of market research) before and after our action, with roughly a cohort of 1000 people, to see if what we did actually managed to change public opinion around animal farming. I’ve never done something like this before but I’m sure there’s many people in EA who would have some tips on how to conduct this research in the best way possible. I think this research would be useful as there hasn’t been much quantitative or rigorous evidence on the impact of protests so I think it would be an interesting experiment generally.
My request: would anyone be interested in having a 30-min video call with me to offer some advice on the best ways to conduct this? Feel free to message me via the EA Forum or email me at james.ozden@hotmail.com. Thanks!
Is there some tension between population ethics + hedonic utilitarianism and the premises people in wild animal suffering use (e.g., negative utilitarianism, or the negative welfare expectancy of wild animals) to argue against rewilding (and in favor of environment destruction)?
If wild animals have bad lives on net, then indiscriminately increasing wild animal populations is bad under any plausible theory of population ethics.
Obviously. But then, first, Effective Environmentalists are doing great harm, right? We should be arguing more about it.
On the other hand, if your basic welfare theory is hedonistic (at least for animals), then one good long life compensates for thousands of short miserable ones—because what matters is qualia, not individuals. And though I don’t deny animals suffer all the time, I guess their “default welfare setting” must be positive if their reward system (at least for vertebrates) is to function properly.
So I guess it’s more likely that we have some sort of instance of the “repugnant conclusion” here.
Ofc, this doesn’t imply we shouldn’t intervene on wild environments to reduce suffering or increase happiness. What is at stake is: U(destroying habitats) > U(restoring habitats)
Does your feeling that the default state is positive also apply to farm animals? Their reward system would be shaped by aritifical selection for the past few generations, but it is not immediately clear to me if you think that would make a difference.
First, it’s not a feeling, it’s a hypothesis. Please, do not mistake one for the other.
It could apply to them if they were not observed to be under stress conditions and captivity, and in behaviors consistent with psychological suffering—like neurotic ticks, vocalization or apathy.
(Tbh, I don’t quite see your point here, but I guess you possibly don’t see mine, either)
This is something interesting that I’ve been thinking about too, as someone who identifies as an environmentalist and who cares about animals. I would say most mainstream environmentalists promote rewilding but it’s not that common with Effective Environmentalism from what I’ve seen so far. You might say it gets lumped in with afforestation but that isn’t exactly rewilding nor that popular within EE anyway. Certainly the issue of more wild animal suffering is one I’ve raised when talking to less-EA aligned folks about rewilding and that’s not gone down well but I haven’t seen it discussed much in EE spaces.
Good point, thanks. However, even if EE and Wild animals welfare advocates do not conflict in their intermediary goals, their ultimate goals do collide, right? For the former, habitat destruction is an evil, and habitat restoration is good—even if it’s not immediately effective.
Possible research/forecasting questions to understand the economic value of AGI research
A common narrative about AI research is that we are on a path to AGI, in that society will be motivated to try to create increasingly general AI systems, culminating in AGI. Since this is a core assumption of the AGI risk hypothesis, I think it’s very important to understand whether this is actually the case.
Some people have predicted that AI research funding will dry up someday as the costs start to outweigh the benefits, resulting in an “AI winter.” Jeff Bigham wrote in 2019 that the AI field will experience an “AI autumn,” in which the AI research community will shift its focus from trying to develop human-level AI capabilities to developing socially valuable applications of narrow AI.
My view is that an AI winter is unlikely to happen anytime soon (10%), an AI autumn is likely to happen eventually (70%), and continued investment in AGI research all the way to AGI is somewhat unlikely (20%). But I think we can try to understand and predict these outcomes better. Here are some ideas for possibly testable research questions:
How much money will OpenAI make by licensing GPT-3?
How long will it take for the technology behind GPT-2 and GPT-3 (roughly, making generic language models do other language tasks without specific training) to become as economically competitive as similar technologies did after they were invented?
How long will it take for DeepMind and OpenAI to break even?
How do the growth rates of DeepMind and OpenAI’s revenues and expenses compare to those of other corporate research labs throughout history?
Will Alphabet downsize or shut down DeepMind?
Will Microsoft scale back or end its partnership with OpenAI?
Notes:
I don’t know of any other labs actively trying to create AGI.
I have no experience with financial analysis, so I don’t know if these questions are the kind that a financial analyst would actually be able to answer. They could be nonsensical for all I know.
I regularly see people write arguments like “One day, we’ll colonize the galaxy—this shows why working on the far future is so exciting!”
I know the intuition this is trying to trigger is bigger = more impact = exciting opportunity.
The intuition it actually triggers for me is expansion and colonization = trying to build an empire = I should be suspicious of these people and their plans.
Do you consider this intuition to be a reason that people should be wary of making this type of argument? Or maybe specifically avoid the word “colonize”?
Maybe something like “populate the galaxy” would be better, as it emphasizes that there are no native populations whose members would be harmed by space colonization?
NB: I have my own little archive of EA content and I got an alert that several links popped up as dead—typically I would just add it to a task list and move on… but I was surprised to see Joe’ Rogan’s (full) interview with Will Macaskill in 2017 was no longer available on YouTube. So I investigated and found out Rogan recently sold his entire catalog and future episodes to Spotify (for $100 million!). Currently Spotify is removing episodes from other platforms like Apple, Youtube and Vimeo. They’ve also decided to not transfer certain episodes that violate their platform’s rules on content (i.e. it’s controversial or offensive). I was a little alarmed that Will’s interview might be on the cut-list but alas it still exists on Spotify, but you now have to make a (free) account to access it.
This is a good thing to call attention to! One of the reasons we add older EA content to the Forum is to make sure it exists in one more easily accessible place, in case another site or service goes down.
Your independent impression about X is essentially what you’d believe about X if you weren’t updating your beliefs in light of peer disagreement—i.e., if you weren’t taking into account your knowledge about what other people believe and how trustworthy their judgement seems on this topic relative to yours. Your independent impression can take into account the reasons those people have for their beliefs (inasmuch as you know those reasons), but not the mere fact that they believe what they believe.
Armed with this concept, I try to stick to the following epistemic/discussion norms, and think it’s good for other people to do so as well:
Trying to keep track of my own independent impressions separately from my all-things-considered beliefs (which also takes into account peer disagreement)
Trying to be clear about whether I’m reporting my independent impression or my all-things-considered belief
Feeling comfortable reporting my own independent impression, even when I know it differs from the impressions of people with more expertise in a topic
In contrast, when I actually make decisions, I try to make them based on my all-things-considered beliefs.
For example, my independent impression is that it’s plausible that a stable, global authoritarian regime, or some other unrecoverable dystopia, is more likely than extinction, and that we should prioritise those risks more than we currently do. But I think that this opinion is probably uncommon among people who’ve thought a lot about existential risks. And that makes me somewhat less confident in this opinion and somewhat less likely to actually act on it. But I think it’s still useful for me to keep track of my independent impression and report it sometimes, or else the community might end up with overly certain and overly homogenous beliefs.
This term and concept and these suggested norms aren’t at all original to me—see in particular Naming beliefs and several of the posts tagged Epistemic humility (especially this one). But I wanted a clear, concise description of this specific set of terms and norms so that I could link to it whenever I say I’m reporting my independent impression, ask someone for theirs, or ask someone whether an opinion they’ve given is their independent impression or their all-things-considered belief.
I just re-read this comment by Claire Zabel, which is also good and is probably where I originally encountered the “impressions” vs “beliefs” distinction.
(Though I still think that this shortform serves a somewhat distinct purpose, in that it jumps right to discussing that distinction, uses terms I think are a bit clearer—albeit clunkier—than just “impressions” vs “beliefs”, and explicitly proposes some discussion norms that Claire doesn’t quite explicitly propose.)
We would like to draw your attention to a new conference on using methods from psychology and other behavioral sciences to understand and promote effective altruism. We invite you to submit an abstract for a talk, poster, or discussion session on a relevant topic by April 25.
The inaugural Life Improvement Science conference will be held online from June 9-13 2021. We invite short abstracts for talks, posters, symposia, and panel discussions on various topics relevant to understanding and promoting the effective pursuit of prosocial values, optimal personal development (moral learning and cognitive growth), and reducing unethical behavior.
Please submit your abstract here. The abstract submission deadline is April 25.
Relevant topics include prosocial behavior and motivation, moral psychology, improving human decision-making and rationality, effective altruism, positive psychology, behavior change, (digital) interventions, character education, environmental psychology, political psychology, behavioral economics and public policy, wisdom scholarship, computational psychiatry, psychotherapy and coaching, intentional personality change, human-centered design and positive computing, cognitive augmentation, moral philosophy, virtues, value change, and other topics.
Our confirmed speakers include David Reinstein, William Fleeson, Ken Sheldon, Brian Little, Igor Grossman, Kristján Kristjánsson, and Kendall Bronk.
If you would like to learn more about LIS and the upcoming LIS conference, you can check out our website. If you would like to stay up to date on Life Improvement Science and the LIS conference, you can sign up for the conference newsletter here or follow us on Twitter and we will keep you posted.
Please help us spread the word about LIS and the LIS conference by forwarding this email to other people who might be interested.
I think empirical claims can be discriminatory. I was struggling with how to think about this for a while, but I think I’ve come to two conclusions. The first way I think that empirical claims can be discrimory is if they express discriminatory claims with no evidence, and people refusing to change their beliefs based on evidence. I think the other way that they can be discriminatory is when talking about the definitions of socially constructed concepts where we can, in some sense and in some contexts, decide what is true.
Are there any EAA researchers carefully tracking the potential of huge cost-effectiveness gains in the ag industry from genetic engineering advances of factory farmed animals? Or (less plausibly) advances from better knowledge/practice/lore from classical artificial selection? As someone pretty far away from the field, a priori the massive gains made in biology/genetics in the last few decades seems like something that we plausibly have not priced in in. So it’d be sad if EAAs get blindsided by animal meat becoming a lot cheaper in the next few decades (if this is indeed viable, which it may not be).
This post may be of interest, in case you haven’t seen it already.
Yep, aware of this! Solid post.
A moral philosophy free-for-all
I wrote this last Summer as a private “blog post” just for me. I’m posting it publicly now (after mild editing) because I have some vague idea that it can be good to make things like this public. These rambling thoughts come from my very naive point of view (as it was in the Summer of 2020; not to suggest my present day point of view is much less naive). In particular if you’ve already read lots of moral philosophy you probably won’t learn anything from reading this.
The free-for-all
Generally, reading various moral philosophy writings has probably made me (even) more comfortable trusting my own intuitions / reasoning regarding what “morality” is and what the “correct moral theory” is.
I think that, when you start engaging with moral philosophy, there’s a bit of a feeling that when you’re trying to reason about things like what’s right and wrong, which moral theory is superior to the other, etc, there are some concrete rules you need to follow, and (relatedly) certain words or phrases have a solid, technical definition that everyone with sufficient knowledge knows and agrees on. The “certain words or phrases” I have in mind here are things like “morally right”, “blameworthy”, “ought”, “value”, “acted wrongly”, etc.
To me right now, the situation seems a bit more like the following: moral philosophers (including knowledgeable amateurs, etc) have in mind definitions for certain words, but these definitions may be more or less precise, might change over time, and differ from person to person. And in making a “moral philosophy” argument (say, writing down an argument for a certain moral theory), the philosopher can use flexibility of interpretation as a tool to make their argument appear more forceful than it really is. Or, the philosopher’s argument might imply that certain things are self-evidently true, and the reader might be (maybe unconsciously) fooled into thinking that this is the case, when in fact it isn’t.
It seems to me now that genuinely self-evident truths are in very short supply in moral philosophy. And, now that I think this is the case, I feel like I have much more licence to make up my own mind about things. That feels quite liberating.
But it does also feel potentially dangerous. Of course, I don’t think it’s dangerous that *I* have freedom to decide what “doing good” means to me. But I might find it dangerous that others have that freedom. People can consider committing genocide to be “doing what is right” and it would be nice to have a stronger argument against this than “this conflicts with my personal definition of what good is”. And, of course, others might well think it’s dangerous that I have the freedom to decide what doing good means.
What does morality even mean?
Now that we’re in this free-for-all, even defining morality seems problematic.
I suppose I can make some observations about myself, like
When I see injustice in the world, I feel a strong urge to do something about it
When I see others suffering, I want to relieve that suffering
I have a strong intuition that it’s conscious experience that ultimately matters—“what you don’t know can’t hurt you” is, I think, literally true
And some conscious experiences are clearly very bad (and some are clearly very good)
And so on
I guess these things are all in the region of “wanting to improve the lives of others”. This sounds a lot like wanting to do what is morally good / morally praiseworthy, and seems at least closely related to morality.
In some ways, whether I label some of my goals and beliefs as being to do with “morality” doesn’t matter—either way, it seems clear that the academic field of moral philosophy is pretty relevant. And presumably when people talk about morality outside of an academic context, they’re at least sometimes talking about roughly the thing I’m thinking of.
A thought on how we describe existential risks from misaligned AI:
Sometimes discussions focus on a fairly specific version of AI risk, which involves humanity being quickly wiped out. Increasingly, though, the emphasis seems to be on the more abstract idea of “humanity losing control of its future.” I think it might be worthwhile to unpack this latter idea a bit more.
There’s already a fairly strong sense in which humanity has never controlled its own future. For example, looking back ten thousand years, no one decided that the sedentary agriculture would increasingly supplant hunting and gathering, that increasingly complex states would arise, that slavery would become common, that disease would take off, that social hierarchies and gender divisions would become stricter, etc. The transition to the modern world, and everything that came with this transition, also doesn’t seem to have been meaningfully chosen (or even really understood by anyone). The most serious effort to describe a possible future in detail — Hanson’s Age of Em — also describes a future with loads of features that most present-day people would not endorse.
As long as there are still strong competitive pressures or substantial random drift, it seems to me, no generation ever really gets to choose the future.[1] It’s actually sort of ambiguous, then, what it means to worry about “losing control of our future.”
Here are a few alternative versions of the concern that feel a bit crisper to me:
If we ‘mess up on AI,’ then even the most powerful individual humans will have unusually little influence over their own lives or the world around them.[2]
If we ‘mess up on AI,’ then future people may be unusually dissatisfied about the world they live in. In other words, people’s preferences will be unfilled to an unusually large degree.
Humanity may have a rare opportunity to take control of its own future, by achieving strong coordination and then locking various things in. But if we ‘mess up on AI,’ then we’ll miss out on this opportunity.[3]
Something that’s a bit interesting about these alternative versions of the concern, though, is that they’re not inherently linked to AI alignment issues. Even if AI systems behave roughly as their users intend, I believe each of these outcomes is still conceivable. For example, if there’s a missed opportunity to achieve strong coordination around AI, the story might look like the failure of the Baruch Plan for international control of nuclear weapons: that failure had much more to do with politics than it had to do with the way engineers designed the technology in question.
In general, if we move beyond discussing very sharp alignment-related catastrophes (e.g. humanity being quickly wiped out), then I think concerns about misaligned AI start to bleed into broader AI governance concerns. It starts to become more ambiguous whether technical alignment issues are actually central or necessary to the disaster stories people tell.
Although, admittedly, notable individuals or groups (e.g. early Christians) do sometimes have a fairly lasting and important influence. ↩︎
As an analogy, in the world of The Matrix, people may not actually have much less control over the long-run future than hunter-gatherers did twenty thousand years ago. But they certainly have much less control over their own lives. ↩︎
Notably, this is only a bad thing if we expect the relevant generation of humans to choose a better future than would be arrived at by default. ↩︎
Do you have the intuition that absent further technological development, human values would drift arbitrarily far? It’s not clear to me that they would—in that sense, I do feel like we’re “losing control” in that even non-extinction AI is enabling a new set of possibilities that modern-day humans would endorse much less than the decisions of future humans otherwise. (It does also feel like we’re missing the opportunity to “take control” and enable a new set of possibilities that we would endorse much more.)
Relatedly, it doesn’t feel to me like the values of humans 150,000 years ago and humans now and even ems in Age of Em are all that different on some more absolute scale.
Certainly not arbitrarily far. I also think that technological development (esp. the emergence of agriculture and modern industry) has played a much larger role in changing the world over time than random value drift has.
I definitely think that’s true. But I also think that was true of agriculture, relative to the values of hunter-gatherer societies.
To be clear, I’m not downplaying the likelihood or potential importance of any of the three crisper concerns I listed. For example, I think that AI progress could conceivably lead to a future that is super alienating and bad.
I’m just (a) somewhat pedantically arguing that we shouldn’t frame the concerns as being about a “loss of control over the future” and (b) suggesting that you can rationally have all these same concerns even if you come to believe that technical alignment issues aren’t actually a big deal.
Wow, I just learned that Robin Hanson has written about this, because obviously, and he agrees with you.
FWIW, I wouldn’t say I agree with the main thesis of that post.
I definitely think that human biology creates at least very strong biases toward certain values (if not hard constraints) and that AI system would not need to have these same biases. If you’re worried about future agents having super different and bad values, then AI is a natural focal point for your worry.
A couple other possible clarifications about my views here:
I think that the outcome of the AI Revolution could be much worse, relative to our current values, than the Neolithic Revolution was relative to the values of our hunter-gatherer ancestors. But I think the question “Will the outcome be worse?” is distinct from the question “Will we have less freedom to choose the outcome?”
I’m personally not so focused on value drift as a driver of long-run social change. For example, the changes associated with the Neolithic Revolution weren’t really driven by people becoming less egalitarian, more pro-slavery, more inclined to hold certain religious beliefs, more ideologically attached to sedentism/farming, more happy to accept risks from disease, etc. There were value changes, but, to some significant degree, they seem to have been downstream of technological/economic change.
Really appreciate the clarifications! I think I was interpreting “humanity loses control of the future” in a weirdly temporally narrow sense that makes it all about outcomes, i.e. where “humanity” refers to present-day humans, rather than humans at any given time period. I totally agree that future humans may have less freedom to choose the outcome in a way that’s not a consequence of alignment issues.
I also agree value drift hasn’t historically driven long-run social change, though I kind of do think it will going forward, as humanity has more power to shape its environment at will.
My impression is that the differences in historical vegetarianism rates between India and China, and especially India and southern China (where there is greater similarity of climate and crops used), is a moderate counterpoint. At the timescale of centuries, vegetarianism rates in India are much higher than rates in China. Since factory farming is plausibly one of the larger sources of human-caused suffering today, the differences aren’t exactly a rounding error.
That’s a good example.
I do agree that quasi-random variation in culture can be really important. And I agree that this variation is sometimes pretty sticky (e.g. Europe being predominantly Christian and the Middle East being predominantly Muslim for more than a thousand years). I wouldn’t say that this kind of variation is a “rounding error.”
Over sufficiently long timespans, though, I think that technological/economic change has been more significant.
As an attempt to operationalize this claim: The average human society in 1000AD was obviously very different than the average human society in 10,000BC. I think that the difference would have been less than half as large (at least in intuitive terms) if there hadn’t been technological/economic change.
I think that the pool of available technology creates biases in the sorts of societies that emerge and stick around. For large enough amounts of technological change, and long enough timespans (long enough for selection pressures to really matter), I think that shifts in these technological biases will explain a large portion of the shifts we see in the traits of the average society.[1]
If selection pressures become a lot weaker in the future, though, then random drift might become more important in relative terms. ↩︎
And Paul Christiano agrees with me. Truly, time makes fools of us all.
Another interpretation of the concern, though related to your (3), is that misaligned AI may cause humanity to lose the potential to control its future. This is consistent with humanity not having (and never having had) actual control of its future; it only requires that this potential exists, and that misaligned AI poses a threat to it.
Something that came up with a discussion with a coworker recently is that often internet writers want some (thoughtful) comments, but not too many, since too many comments can be overwhelming. Or at the very least, the marginal value of additional comments is usually lower for authors when there are more comments.
However, the incentives for commentators is very different: by default people want to comment on the most exciting/cool/wrong thing, so internet posts can easily by default either attract many comments or none. (I think) very little self-policing is done, if anything a post with many comments make it more attractive to generate secondary or tertiary comments, rather than less.
Meanwhile, internet writers who do great work often do not get the desired feedback. As evidence: For ~ a month, I was the only person who commented on What Helped the Voiceless? Historical Case Studies (which later won the EA Forum Prize).
This will be less of a problem if internet communication is primarily about idle speculations and cat pictures. But of course this is not the primary way I and many others on the Forum engage with the internet. Frequently, the primary publication venue for some subset of EA research and thought is this form of semi-informal internet communication.
Academia gets around this problem by having mandated peer review for work, with a minimum number of peer reviewers who are obligated to read (or at least skim) anything that passes through an initial filter.
It is unclear how (or perhaps if?) we ought to replicate something similar in a voluntary internet community.
On a personal level, the main consequence of this realization is that I intend to become slightly more inclined to comment more on posts without many prior comments, and correspondingly less to posts with many comments (especially if the karma:comment ratio is low).
On a group level, I’m unsure what behavior we ought to incentivize, and/or how to get there. Perhaps this is an open question that others can answer?
I think these are useful observations and questions. (Though I think “too many comments” should probably be much less of a worry than “too few”, at least if the comments make some effort to be polite and relevant, and except inasmuch as loads of comments on one thing sucks up time that could be spent commenting on other things where that’d be more useful.)
I think a few simple steps that could be taken by writers are:
People could more often send google doc drafts to a handful of people specifically selected for being more likely than average to (a) be interested in reading the draft and (b) have useful things to say about it
People could more often share google doc drafts in the Effective Altruism Editing & Review Facebook group
People could more often share google doc drafts in other Facebook groups, Slack workspaces, or the like
E.g., sharing a draft relevant to improving institutional decision-making in the corresponding Facebook group
People could more often make posts/shortforms that include an executive summary (or similar) and a link to the full google doc draft, saying that this is still like a draft and they’d appreciate comment
Roughly this has been done recently by Joe Carlsmith and Ben Garfinkel, for example
This could encourage more comments that just posting the whole thing to the Forum as a regular post, since (a) this conveys that this is still a work-in-progress and that comments are welcome, and (b) google docs make it easier to comment on specific points
When people do post full versions of things on the Forum (or wherever), they could explicitly indicate that they’re interested in feedback, indicate roughly what kinds of feedback would be most valuable, and indicate that they might update the post in light of feedback (if that’s true)
People could implement the advice given in these two good posts:
https://forum.effectivealtruism.org/posts/N3zd4FtGmRnMF7pfM/asking-for-advice
https://forum.effectivealtruism.org/posts/ZjiokgANEfu36LD6G/giving-and-receiving-feedback
I think a few simple steps that could be taken by potential reviewers are:
As you suggest, people could adjust their behaviours a bit more towards commenting on posts without many prior comments, and save time to do that by commenting a bit less on posts with many comments
Likewise, people could adjust their behaviours a bit more towards commenting on drafts they come across where the author is seeking feedback (e.g., drafts that were sent to the person directly or shared in some group the person is part of), especially if they don’t yet have many prior comments
People could implement the advice given in https://forum.effectivealtruism.org/posts/ZjiokgANEfu36LD6G/giving-and-receiving-feedback
There are presumably also other options one could come up with. And maybe something more systematic/institutionalised would be good.
One other semi-relevant thing from my post Notes on EA-related research, writing, testing fit, learning, and the Forum:
I’m considering donating to the Centre for Women’s Justice. With a budget of about £300k last year, they have undertaken strategic litigation against the government , Crown prosecutors etc for mismanagement of sexual assault cases. The cases seem well-chosen to raise the issue on the political agenda. I think more rapists being successfully prosecuted would have a very positive impact so I’m excited to see this work. I’m planning to email them soon. https://www.centreforwomensjustice.org.uk/strategic-plan
Some initial thoughts on “Are We Living At The Hinge Of History”?
In the below I give a very rough summary of Will MacAskill’s article Are We Living At The Hinge Of History? and give some very preliminary thoughts on the article and some of the questions it raises.
I definitely don’t think that what I’m writing here is particularly original or insightful: I’ve thought about this for no more than a few days, any points I make are probably repeating points other people have already made somewhere, and/or are misguided, etc. This seems like an incredibly deep topic which I feel like I’ve barely scratched the surface of. Also, this is not a focussed piece of writing trying to make a particular point, it’s just a collection of thoughts on a certain topic.
(If you want to just see what I think, skip down to “Some thoughts on the issues discussed in the article”)
A summary of the article
(note that the article is an updated version of the original EA Forum post Are we living at the most influential time in history?)
Definition for the Hinge of History (HH)
The Hinge of History claim (HH): we are among the most influential people ever (past or future). Influentialness is, roughly, how much good a particular person at a particular time can do through direct expenditure of resources (rather than investment)
Two prominent longtermist EA views imply HH
Two worldviews prominent in longtermist EA imply that HH is true:
Time of Perils view: we live at a time of unusually high extinction risk, and we can do an unusual amount to reduce this risk
Value Lock-In view: we’ll soon invent a technology that allows present-day agents to assert their values indefinitely into the future (in the Bostrom-Yudkowsky version of this view, the technology is AI)
Arguments against HH
The base rates argument
Claim: our prior should be that we’re as likely as anyone else, past or present, to be the most influential person ever (Bostrom’s Self-Sampling Assumption (SSA)). Under this prior, it’s astronomically unlikely that any particular person is the most influential person ever.
Then the question is how much should we update from this prior
The standard of evidence (Bayes factor) required to favour HH is incredibly high. E.g. we need a Bayes factor of ~107 to move from a 1 in 100 million credence to a 1 in 10 credence. For comparison, a p=0.05 result from a randomised controlled trial gives a Bayes factor of 3 under certain reasonable assumptions.
The arguments for Time of Perils or Value Lock-In might be somewhat convincing; but hard to see how they could be convincing enough
E.g. our track record of understanding the importance of historical events is very poor
When considering how much to update from the prior, we should be aware that there are biases that will tend to make us think HH is more likely than it really is
Counterargument 1: we only need to be at an enormously influential time, not the most influential, and the implications are ~the same either way
Counter 1 to counterargument 1: the Bostrom-Yudkowsky view says we’re at the most influential time ever, so you should reject the Bostrom-Yudkowsky view if you’re abandoning the idea that we’re at the most influential time ever. So there is a material difference between “enormously influential time” and “most influential time”.
Counter 2 to counterargument 1: if we’re not at the most influential time, presumably we should transfer our resources forward to the most influential time, so the difference between “enormously influential time” and “most influential time” is highly action-relevant.
Counterargument 2: the action-relevant thing is the influentialness of now compared to any time we can pass resources on to
Again the Bostrom-Yudkowsky view is in conflict with this
But MacAskill concedes that it does seem right that this is the action-relevant thing. So e.g. we could assume we can only transfer resources 1000 years into the future and define Restricted-HH: we are among the most influential people out of the people who will live over the next 1000 years
The inductive argument
Claim: The influentialness of comparable people has been increasing over time, and we should expect this to continue, so the influentialness of future people who we can pass resources onto will be greater
Evidence: if we consider the state of knowledge and ethics in 1600 vs today, or in 1920/1970 vs today, it seems clear that we have more knowledge and better ethics now than we did in 1600 or in 1920/1970
And seems clear that there are huge gaps in our knowledge today (so doesn’t seem that we should expect this trend to break)
Arguments for HH
Single planet
Argument 1: we’re living on a single planet, implying greater influentialness
Implies particular vulnerabilities e.g. asteroid strikes
Implies individual people have an unusually large fraction of total resources
Implies instant global communication
Counterargument:
Asteroids are not a big risk
For other prominent risks like AI or totalitarianism, being on multiple planets doesn’t seem to help
We might well have quite a long future period on earth (1000s or 10,000s of years), which makes being on earth now less special
And in the early stages of space settlement the picture isn’t necessarily that relevantly different to the single planet one
Rapid growth
Argument 2: we’re now in a period of unusually fast economic and tech progress, implying greater influentialness. We can’t maintain the present-day growth rate indefinitely.
MacAskill seems sympathetic to the argument, but says it implies not that today is the most important time, but that the most important time is some time might be in the next few thousand years
Also, maybe longtermist altruists are less influential during periods of fast economic growth because rapid change makes it harder to plan reliably
And comparing economic power across long timescales is difficult
Other arguments
A few other arguments for HH are briefly touched on in a footnote: that existential risk / value lock-in lowers the number of future people in the reference class for the influentialness prior; that we might choose other priors that are more favourable for HH, and that earlier people can causally affect more future people
Some quick meta-level thoughts on the article
I wish it had a detailed discussion about choosing a prior for influentialness, which I think is really important.
There’s a comment that the article ignores the fact that the annual risk of extinction or lock-in in the future has implications for present-day influentialness because in Trammell’s model this is incorporated into the pure rate of time preference. I find that pretty weird. Trammell’s model is barely referenced elsewhere in the paper so I don’t really see why we should neglect to discuss something just because it happens to be interpreted in a certain way within his model. Maybe I missed the point here.
I think it’s a shame that MacAskill doesn’t really give numbers in the article for his prior and posterior, either on HH or restricted-HH (this EA Forum comment thread by Greg Lewis is relevant).
Some thoughts on the issues discussed in the article
Two main points from the article
It kind of feels like there are two somewhat independent things that are most interesting from the article:
1. The claim: we should reject the Time of Perils view, and the Bostrom-Yudkowsky view, because in both cases the implication for our current influentialness is implausible
2. The question: what do high level / relatively abstract arguments tell us about whether we can do the most good by expending resources now or by passing resources on to future generations?
Avoiding rejecting the Time of Perils and Bostrom-Yudkowsky views
I think there are a few ways we can go to avoid rejecting the Time of Perils and Bostrom-Yudkowsky views
We can find the evidence in favour of them strong enough to overwhelm the SSA prior through conventional Bayesian updating
We can find the evidence in favour of them weaker than in the previous case, but still strong enough that we end up giving them significant credence in the face of the SSA prior, through some more forgiving method than Bayesian updating
We can use a different prior, or claim that we should be uncertain between different priors
Or we can just turn the argument (back?) around, and say that the SSA prior is implausible because it implies such a low probability for the Time of Perils and Bostrom-Yudkowsky views. Toby Ord seems to say something like this in the comments to the EA Forum post (see point 3).
A nearby alternative is to modify the Time of Perils and Bostrom-Yudkowsky views a bit so that they don’t imply we’re among the most influential people ever. E.g. for the Bostrom-Yudkowsky view we could make the value lock-in a bit “softer” by saying that for some reason, not necessarily known/stated, the lock-in would probably end after some moderate (on cosmological scales) length of time. I’d guess that many people might find a modified view more plausible even independently of the influentialness implications.
I’m not really sure what I think here, but I feel pretty sympathetic to the idea that we should be uncertain about the prior and that this maybe lends itself to having not too strong a prior against the Time of Perils and Bostrom-Yudkowsky views.
On the question of whether to expend resources now or later
The arguments MacAskill discusses suggest that the relevant time frame is the next few thousand years (because the next few thousand years seem (in expectation) especially high influentialness and because it might be effectively impossible to pass our resources further into the future).
It seems like the pivotal importance of priors on influentialness (or similar) then evaporates: it no longer seems that implausible on the SSA prior that now is a good time to expend resources rather than save. E.g. say there’ll be a 20 year period in the next 1000 years where we want to expend philanthropic resources rather than save them to pass on to future generations. Then a reasonable prior might be that we have a 20/1000 = 1 in 50 chance of being in that period. That’s a useful reference point and is enough to make us skeptical about arguments that we are in such a period, but it doesn’t seem overwhelming. In fact, we’d probably want to spend at least some resources now even purely based on this prior.
In particular, it seems like some kind of detailed analysis is needed, maybe along the lines of Trammell’s model or at least using that model as a starting point. I think many of the arguments in MacAskill’s article should be part of that detailed analysis, but, to stress the point, they don’t seem decisive to me.
This comment by Carl Shulman on the related EA Forum post and its replies has some stuff on this.
The importance of the idea of moral progress
In the article, the Inductive Argument is supported by the idea of moral progress: MacAskill cites the apparent progress in our moral values over the past 400 years as evidence for the idea that we should expect future generations to have better moral values than we do. Obviously, whether we should expect moral progress in the future is a really complex question, but I’m at least sympathetic to the idea that there isn’t really moral progress, just moral fashions (so societies closer in time to ours seem to have better moral values just because they tend to think more like us).
Of course, if we don’t expect moral progress, maybe it’s not so surprising that we have very high influentialness: if past and future actors don’t share our values, it seems very plausible on the face of it that we’re better off expending our resources now than passing them off to future generations in the hope they’ll carry out our wishes. So maybe MacAskill’s argument about influentialness should update us away from the idea of moral progress?
This EA Forum comment by Larks and this one by Kbog are relevant.
But if we’re steadfast in our belief in moral progress, maybe it’s not so surprising that we have high influentialness because we find ourselves in a world where we are among the very few with a longtermist worldview, which won’t be the case in the future as longtermism becomes a more popular view. (I think Carl Shulman might say something like this in the comments to the original EA Forum post)
My overall take
I think “how plausible is this stuff under an SSA prior” is a useful perspective
Still, thinking about this hasn’t caused me to completely dismiss the Time of Perils View or the Bostrom-Yudkowsky view (I probably already had some kind of strong implausibility prior on those views).
The arguments in the article are useful for thinking about how much (e.g.) the EA longtermist community should be spending rather than saving now, but a much more detailed analysis seems necessary to come to a firm view on this.
A quote to finish
I like the way the article ends, providing some motivation for the Inductive Argument in a way I find appealing on a gut level:
Just as our powers to grow crops, to transmit information, to discover the laws of nature, and to explore the cosmos have all increased over time, so will our power to make the world better — our influentialness. And given how much there is still to understand, we should believe, and hope, that our descendents look back at us as we look back at those in the medieval era, marvelling at how we could have got it all so wrong.
Some initial thoughts on moral realism vs anti-realism
I wrote this last Summer as a private “blog post” just for me. I’m posting it publicly now (after mild editing) because I have some vague idea that it can be good to make things like this public. These thoughts come from my very naive point of view (as it was in the Summer of 2020; not to suggest my present day point of view is much less naive). In particular if you’ve already read lots of moral philosophy you probably won’t learn anything from reading this. Also, I hope my summaries of other people’s arguments aren’t too inaccurate.
Recently, I’ve been trying to think seriously about what it means to do good. A key part of Effective Altruism is asking ourselves how we can do the most good. Often, considering this question seems to be mostly an empirical task: how many lives will be saved through intervention A, and how many through intervention B? Aside from the empirical questions, though, there are also theoretical ones. One key consideration is what we mean by doing good.
Moral Philosophy
There is a branch of philosophy called moral philosophy which is (partly) concerned with answering this question.
It’s important to me that I don’t get too drawn into the particular framings that have evolved within the academic discipline of moral philosophy, which are, presumably, partly due to cultural or historical forces, etc. This is because I really want to try to come up with my own view, and I think that (for me) the best process for this involves not taking other people’s views or existing works too seriously, especially while I try to think about these things seriously for the first time.
Still, it seems useful to get familiar with the major insights and general way of thinking within moral philosophy, because
I’ll surely learn a lot of useful stuff
I’ll be able to communicate with other people who are familiar with moral philosophy (which probably includes most of the most interesting people to talk to on this topic).
Moral realism
I’ve read a couple of Stanford Encyclopedia of Philosophy articles, and a series of posts by Lukas Gloor arguing for moral anti-realism.
I found the Stanford Encyclopedia of Philosophy articles fairly tough going but still kind of useful. I thought the Gloor posts were great.
The Gloor posts have kind of convinced me to take the moral anti-realist side, which, roughly, denies the existence of definitive moral truths.
While I suppose I might consider my “inside view” to be moral anti-realist at the moment, I can easily see this changing in the future. For example, I imagine that if I read a well-argued case for moral realism, I might well change my mind.
In fact, prior to reading Gloor’s posts, I probably considered myself to be a moral realist. I think I’d heard arguments, maybe from Will MacAskill, along the lines that i) if moral anti-realism is true, then nothing matters, whereas if realism is true, you should do what the true theory requires you to do, and ii) there’s some chance that realism is true, therefore iii) you should do what the true theory requires you to do.
Gloor discusses an argument like this in one of his posts. He calls belief in moral realism founded on this sort of argument “metaethical fanaticism” (if I’ve understood him correctly).
I’m not sure that I completely understood everything in Gloor’s posts. But the “fanaticism” label does feel appropriate to me. It feels like there’s a close analogy with the kinds of fanaticism that utilitarianism is susceptible to, for example. An example of that might be a Pascal’s wager type argument—if there’s a finite probability that I’ll get infinite utility derived from an eternal life in a Christian heaven, I should do what I can to maximise that probability.
It feels like something has gone wrong here (although admittedly it’s not clear what), and this Pascal’s wager argument doesn’t feel at all like a strong argument for acting as if there’s a Christian heaven. Likewise, the “moral realist wager” doesn’t feel like a strong argument for acting as if moral realism is true, in my current view.
Moral anti-realism
Gloor also argues that we don’t lose anything worth having by being moral anti-realists, at least if you’re his brand of moral anti-realist. I think he calls the view he favours “pluralistic moral reductionism”.
On his view, you take any moral view (or maybe combination of views) you like. These can (and maybe for some people, “should”) be grounded in our moral intuitions, and maybe use notions of simplicity of structure etc, just as a moral realist might ground their guess(?) at the true moral theory in similar principles. Your moral view is then your own “personal philosophy”, which you choose to live by.
One unfortunate consequence of this view is that you don’t really have any grounds to argue with someone else who happens to have a different view. Their view is only “wrong” in the sense that it doesn’t agree with yours; there’s no objective truth here.
From this perspective, it would arguably be nicer if everyone believed that there was a true moral view that we should strive to follow (even if we don’t know what it is). Especially if you also believe that we could make progress towards that true moral view.
I’m not sure how big this effect is, but it feels like more than nothing. So maybe I don’t quite agree that we don’t lose anything worth having by being moral anti-realists.
In any case, the fact that we might wish that moral realism is true doesn’t (presumably) have any bearing on whether or not it is true.
Reflections
I already mentioned that reading Gloor’s posts has caused me to favour moral anti-realism. Another effect, I think, is that I am more agnostic about the correct moral theory. Some form of utilitarianism, or at least consequentialism, seems far more plausible to me as the moral realist “one true theory” than a deontological theory or virtue ethics theory. Whereas if moral anti-realism is correct, I might be more open to non-consequentialist theories. (I’m not sure whether this new belief would stand up to a decent period of reflection, though—maybe I’d be just as much of a convinced moral anti-realist consequentialist after some reflection).
A thought on epistemic deference:
The longer you hold a view, and the more publicly you hold a view, the more calcified it typically becomes. Changing your mind becomes more aversive and potentially costly, you have more tools at your disposal to mount a lawyerly defense, and you find it harder to adopt frameworks/perspectives other than your favored one (the grooves become firmly imprinted into your brain). At least, this is the way it seems and personally feels to me.[1]
For this reason, the observation “someone I respect publicly argued for X many years ago and still believes X” typically only provides a bit more evidence than the observation “someone I respect argued for X many years ago.” For example, even though I greatly respect Daron Acemoglu, I think the observation “Daron Acemoglu still believes that political institutions are the central determinant of economic growth rates” only gives me a bit more evidence than the observation “15 years ago Daron Acemoglu publicly argued that institutions are the central determinant of economic growth rates.”
A corollary: If there’s an academic field that contains a long-standing debate, and you’d like to defer to experts in this field, you may want to give disproportionate weight to the opinions of junior academics. They’re less likely to have responded to recent evidence and arguments in an epistemically inflexible way.
Of course, there are exceptions. The final chapter of Scout Mindset includes a moving example of a professor publicly abandoning a view he had championed for fifteen years, after a visiting academic presented persuasive new evidence. The reason these kinds of stories are moving, though, is that they describe truly exceptional behavior. ↩︎
At least in software, there’s a problem I see where young engineers are often overly bought-in to hype trains, but older engineers (on average) stick with technologies they know too much.
I would imagine something similar in academia, where hot new theories are over-valued by the young, but older academics have the problem you describe.
Good point!
That consideration—and the more basic consideration that more junior people often just know less—definitely pushes in the opposite direction. If you wanted to try some version of seniority-weighted epistemic deference, my guess is that the most reliable cohort would have studied a given topic for at least a few years but less than a couple decades.
Previously, I was wondering what rather high rates of mental health issues imply about philosophical positions EA’s are attracted to. Now I know that according to The psychology of philosophy: Associating philosophical views with psychological traits in professional philosophers “lower levels of well-being and higher levels of mental illness predicted hard determinism.” I am highly uncertain but it doesn’t seem that significant to me.
h/t Gavin
Write roundup posts!
The posts I’ve made that I think yielded the most value for the amount of work I put in were essentially lists of other people’s work.
EA Syllabi and teaching materials
Giving now vs. later: a summary
There are other formats that may make sense, like tags for material on this forum, or wikis. But the general principle is that you can do something really useful by making it easy for people to find existing material on a topic.
I was thinking about Chad current situation and Vox’s piece on Parliamentarism… Has anyone assessed (ITN or EA-style evaluation) political stability as a cause area?
I mean, it’s pretty relevant for peace (I guess most wars result from conflict of factions or succession crises) and for a well functioning government. People talk about the dangers of polarization, about why nations fail, or autoritarianism, or iidm… It’s not neglected per se (it’s been the focus of some of classical works in political phil & sci), but I’m not sure all low-hanging has been eaten; plus, thinking about interventions as increasing / decreasing political stability might help assessing other areas (like IIDM).
Quick thoughts on Kelsey Piper’s article Is climate change an “existential threat” — or just a catastrophic one?
The article was far better than I expect most reporting on climate change as a potential existential risk to be
This is in line with Kelsey Piper generally seeming to do great work
I particularly appreciated that it (a) emphasised how the concepts of catastrophes in general and extinction in particular are distinct and why that matters, but (b) did this in a way that I suspect has a relatively low risk of seeming callous, nit-picky, or otherwise annoying to people who care about climate change
But I also had some substantive issues with the article, which I’ll discuss below
The article conflated “existential threat”/“existential risk” with “extinction risk”, thereby ignoring two other types of existential catastrophe: unrecoverable collapse and unrecoverable dystopia
See also Venn diagrams of existential, global, and suffering catastrophes
Some quotes from the article to demonstrate what the conflation I’m referring to:
“But there’s a standard meaning of that phrase [existential threat]: that it’s going to wipe out humanity — or even, as Warren implied Wednesday night, all life on our planet.”
“To academics in philosophy and public policy who study the future of humankind, an existential risk is a very specific thing: a disaster that destroys all future human potential and ensures that no generations of humans will ever leave Earth and explore our universe.”
I think this also means the article kind-of ignores or overconfidently dismisses the possibility that climate change might cause unrecoverable collapse or unrecoverable dystopia
(The article does mention collapse a few times, but not something that corresponds to unrecoverable collapse or unrecoverable dystopia)
I do think it is very unlikely (e.g., below 1 in 1000 chance) that climate change would relatively directly cause those things, but that’s only a tentative view, and I think uncertainty and further research is warranted
I expect that the fact that article kind-of ignores unrecoverable collapse would cause some non-EA-types to disagree with the article, and that they’d be right to do so
(Though I also expect that many of these people would be overconfident that climate change would cause a major collapse, that they’d pay insufficient attention to the question of whether this collapse is “unrecoverable”, and that they also wouldn’t consider reasonable scenarios of dystopias)
The article also failed to mention the term “existential risk factor” or allude to that basic idea
It thus ignores or overconfidently dismisses the possibility that climate change might make x-risks more likely, even if it doesn’t directly cause them
Finally, the article failed to mention one other, important reason why the distinction between catastrophes in general and extinction in particular matters: This distinction is relevnat to prioritisation, in a world where other things may indeed be plausible extinction risks
If people care about climate change because they think it’s fairly likely to cause extinction, but in reality it’s less likely to cause extinction and something else is more likely to do so, then (in theory!), by their own values, it should be really important for them to learn that and adjust their priorities
But this is more just something I think could’ve maybe made the article even better, rather than something that I felt was problematic about it
I think it’s totally understandable that Kelsey Piper didn’t discuss the above points in detail, but I think she could’ve:
Been more careful in her phrasing to at least avoid being actively misleading
Maybe briefly touched on (some) of these points and provided links to where they’re discussed more thoroughly
(Disclaimer-ish thing: I haven’t sent this to Kelsey because it doesn’t seem super important, the article is from 2019, I assume she’s quite busy, and I’m posting this as a shortform rather than something super prominent.)
Small request for help to measure the impact of an intervention we have planned:
In short, Animal Rebellion UK is planning a large protest/civil resistance action and I’m keen to try quantitatively and semi-rigorously measure the impact of this, budget permitting. We’re planning on doing an opinion poll (or another bit of market research) before and after our action, with roughly a cohort of 1000 people, to see if what we did actually managed to change public opinion around animal farming. I’ve never done something like this before but I’m sure there’s many people in EA who would have some tips on how to conduct this research in the best way possible. I think this research would be useful as there hasn’t been much quantitative or rigorous evidence on the impact of protests so I think it would be an interesting experiment generally.
My request: would anyone be interested in having a 30-min video call with me to offer some advice on the best ways to conduct this? Feel free to message me via the EA Forum or email me at james.ozden@hotmail.com. Thanks!
Is there some tension between population ethics + hedonic utilitarianism and the premises people in wild animal suffering use (e.g., negative utilitarianism, or the negative welfare expectancy of wild animals) to argue against rewilding (and in favor of environment destruction)?
If wild animals have bad lives on net, then indiscriminately increasing wild animal populations is bad under any plausible theory of population ethics.
Obviously. But then, first, Effective Environmentalists are doing great harm, right? We should be arguing more about it. On the other hand, if your basic welfare theory is hedonistic (at least for animals), then one good long life compensates for thousands of short miserable ones—because what matters is qualia, not individuals. And though I don’t deny animals suffer all the time, I guess their “default welfare setting” must be positive if their reward system (at least for vertebrates) is to function properly. So I guess it’s more likely that we have some sort of instance of the “repugnant conclusion” here. Ofc, this doesn’t imply we shouldn’t intervene on wild environments to reduce suffering or increase happiness. What is at stake is: U(destroying habitats) > U(restoring habitats)
Does your feeling that the default state is positive also apply to farm animals? Their reward system would be shaped by aritifical selection for the past few generations, but it is not immediately clear to me if you think that would make a difference.
First, it’s not a feeling, it’s a hypothesis. Please, do not mistake one for the other. It could apply to them if they were not observed to be under stress conditions and captivity, and in behaviors consistent with psychological suffering—like neurotic ticks, vocalization or apathy. (Tbh, I don’t quite see your point here, but I guess you possibly don’t see mine, either)
This is something interesting that I’ve been thinking about too, as someone who identifies as an environmentalist and who cares about animals. I would say most mainstream environmentalists promote rewilding but it’s not that common with Effective Environmentalism from what I’ve seen so far. You might say it gets lumped in with afforestation but that isn’t exactly rewilding nor that popular within EE anyway. Certainly the issue of more wild animal suffering is one I’ve raised when talking to less-EA aligned folks about rewilding and that’s not gone down well but I haven’t seen it discussed much in EE spaces.
Good point, thanks. However, even if EE and Wild animals welfare advocates do not conflict in their intermediary goals, their ultimate goals do collide, right? For the former, habitat destruction is an evil, and habitat restoration is good—even if it’s not immediately effective.
Possible research/forecasting questions to understand the economic value of AGI research
A common narrative about AI research is that we are on a path to AGI, in that society will be motivated to try to create increasingly general AI systems, culminating in AGI. Since this is a core assumption of the AGI risk hypothesis, I think it’s very important to understand whether this is actually the case.
Some people have predicted that AI research funding will dry up someday as the costs start to outweigh the benefits, resulting in an “AI winter.” Jeff Bigham wrote in 2019 that the AI field will experience an “AI autumn,” in which the AI research community will shift its focus from trying to develop human-level AI capabilities to developing socially valuable applications of narrow AI.
My view is that an AI winter is unlikely to happen anytime soon (10%), an AI autumn is likely to happen eventually (70%), and continued investment in AGI research all the way to AGI is somewhat unlikely (20%). But I think we can try to understand and predict these outcomes better. Here are some ideas for possibly testable research questions:
What will be the ROI on:
Microsoft’s $1 billion dollar investment and partnership with OpenAI?
Microsoft’s GPT-3 licensing deal with OpenAI?
Google/Alphabet’s acquisition of DeepMind?
How much money will OpenAI make by licensing GPT-3?
How long will it take for the technology behind GPT-2 and GPT-3 (roughly, making generic language models do other language tasks without specific training) to become as economically competitive as similar technologies did after they were invented?
How long will it take for DeepMind and OpenAI to break even?
How do the growth rates of DeepMind and OpenAI’s revenues and expenses compare to those of other corporate research labs throughout history?
Will Alphabet downsize or shut down DeepMind?
Will Microsoft scale back or end its partnership with OpenAI?
Notes:
I don’t know of any other labs actively trying to create AGI.
I have no experience with financial analysis, so I don’t know if these questions are the kind that a financial analyst would actually be able to answer. They could be nonsensical for all I know.
Have you seen the Metaculus “Forecasting AI Progress” tournament?
Nope!
I regularly see people write arguments like “One day, we’ll colonize the galaxy—this shows why working on the far future is so exciting!”
I know the intuition this is trying to trigger is bigger = more impact = exciting opportunity.
The intuition it actually triggers for me is expansion and colonization = trying to build an empire = I should be suspicious of these people and their plans.
Do you consider this intuition to be a reason that people should be wary of making this type of argument? Or maybe specifically avoid the word “colonize”?
Maybe something like “populate the galaxy” would be better, as it emphasizes that there are no native populations whose members would be harmed by space colonization?
Or “fill the universe/galaxy with life”.
NB: I have my own little archive of EA content and I got an alert that several links popped up as dead—typically I would just add it to a task list and move on… but I was surprised to see Joe’ Rogan’s (full) interview with Will Macaskill in 2017 was no longer available on YouTube. So I investigated and found out Rogan recently sold his entire catalog and future episodes to Spotify (for $100 million!). Currently Spotify is removing episodes from other platforms like Apple, Youtube and Vimeo. They’ve also decided to not transfer certain episodes that violate their platform’s rules on content (i.e. it’s controversial or offensive). I was a little alarmed that Will’s interview might be on the cut-list but alas it still exists on Spotify, but you now have to make a (free) account to access it.
This is a good thing to call attention to! One of the reasons we add older EA content to the Forum is to make sure it exists in one more easily accessible place, in case another site or service goes down.
Independent impressions
Your independent impression about X is essentially what you’d believe about X if you weren’t updating your beliefs in light of peer disagreement—i.e., if you weren’t taking into account your knowledge about what other people believe and how trustworthy their judgement seems on this topic relative to yours. Your independent impression can take into account the reasons those people have for their beliefs (inasmuch as you know those reasons), but not the mere fact that they believe what they believe.
Armed with this concept, I try to stick to the following epistemic/discussion norms, and think it’s good for other people to do so as well:
Trying to keep track of my own independent impressions separately from my all-things-considered beliefs (which also takes into account peer disagreement)
Trying to be clear about whether I’m reporting my independent impression or my all-things-considered belief
Feeling comfortable reporting my own independent impression, even when I know it differs from the impressions of people with more expertise in a topic
One rationale for that bundle of norms is to avoid information cascades.
In contrast, when I actually make decisions, I try to make them based on my all-things-considered beliefs.
For example, my independent impression is that it’s plausible that a stable, global authoritarian regime, or some other unrecoverable dystopia, is more likely than extinction, and that we should prioritise those risks more than we currently do. But I think that this opinion is probably uncommon among people who’ve thought a lot about existential risks. And that makes me somewhat less confident in this opinion and somewhat less likely to actually act on it. But I think it’s still useful for me to keep track of my independent impression and report it sometimes, or else the community might end up with overly certain and overly homogenous beliefs.
This term and concept and these suggested norms aren’t at all original to me—see in particular Naming beliefs and several of the posts tagged Epistemic humility (especially this one). But I wanted a clear, concise description of this specific set of terms and norms so that I could link to it whenever I say I’m reporting my independent impression, ask someone for theirs, or ask someone whether an opinion they’ve given is their independent impression or their all-things-considered belief.
I just re-read this comment by Claire Zabel, which is also good and is probably where I originally encountered the “impressions” vs “beliefs” distinction.
(Though I still think that this shortform serves a somewhat distinct purpose, in that it jumps right to discussing that distinction, uses terms I think are a bit clearer—albeit clunkier—than just “impressions” vs “beliefs”, and explicitly proposes some discussion norms that Claire doesn’t quite explicitly propose.)
We would like to draw your attention to a new conference on using methods from psychology and other behavioral sciences to understand and promote effective altruism. We invite you to submit an abstract for a talk, poster, or discussion session on a relevant topic by April 25.
The inaugural Life Improvement Science conference will be held online from June 9-13 2021. We invite short abstracts for talks, posters, symposia, and panel discussions on various topics relevant to understanding and promoting the effective pursuit of prosocial values, optimal personal development (moral learning and cognitive growth), and reducing unethical behavior.
Please submit your abstract here. The abstract submission deadline is April 25.
Relevant topics include prosocial behavior and motivation, moral psychology, improving human decision-making and rationality, effective altruism, positive psychology, behavior change, (digital) interventions, character education, environmental psychology, political psychology, behavioral economics and public policy, wisdom scholarship, computational psychiatry, psychotherapy and coaching, intentional personality change, human-centered design and positive computing, cognitive augmentation, moral philosophy, virtues, value change, and other topics.
Our confirmed speakers include David Reinstein, William Fleeson, Ken Sheldon, Brian Little, Igor Grossman, Kristján Kristjánsson, and Kendall Bronk.
If you would like to learn more about LIS and the upcoming LIS conference, you can check out our website. If you would like to stay up to date on Life Improvement Science and the LIS conference, you can sign up for the conference newsletter here or follow us on Twitter and we will keep you posted.
Please help us spread the word about LIS and the LIS conference by forwarding this email to other people who might be interested.
I suggest publishing this as a post, rather than in the shortform. Is there any particular reason you chose not to?
I think empirical claims can be discriminatory. I was struggling with how to think about this for a while, but I think I’ve come to two conclusions. The first way I think that empirical claims can be discrimory is if they express discriminatory claims with no evidence, and people refusing to change their beliefs based on evidence. I think the other way that they can be discriminatory is when talking about the definitions of socially constructed concepts where we can, in some sense and in some contexts, decide what is true.