I (re)started EA at the University of Washington in early 2022, and I am also preparing for a career in AI alignment.
Alex Mallen
Karma: 165
This post by Rohin attempts to address it. If you hold the asymmetry view then you would allocate more resources to [1] causing a new neutral life to come into existence (-1 cent) then later once they exist improve that neutral life (many dollars) than you would to [2] causing a new happy life to come into existence (-1 cent). They both result in the same world.
In general you can make a dutch booking argument like this whenever your resource allocation doesn’t correspond to the gradient of a value function (i.e. the resources should be aimed at improving the state of the world).
Thank you for pointing me to that and getting me to think critically about it. I think I agree with all the axioms.
a rational agent should act as to maximize expected value of their value function
I think this is misleading. The VNM theorem only says that there exists a function such that a rational agent’s actions maximize . But does not have to be “their value function.”
Consider a scenario in which there are 3 possible outcomes: = enormous suffering, = neutral, = mild joy. Let’s say my value function is , and , in the intuitive sense of the word “value.”
When I work through the proof you sent in this example, I am forced to prefer for some probability , but this probability does not have to be 0.1, so I don’t have to maximize my expected value. In reality, I would be “risk averse” and assign or something. See 4.1Automatic consideration of risk aversion.
More details of how I filled in the proof:
We normalize my value function so and . Then we define .
Let , then , and I am indifferent between and. However, nowhere did I specify what is, so “there exists a function u such that I’m maximizing the expectation of it” is not that meaningful, because it does not have to align with the value I assign to the event.
I’m concerned about getting involved in politics on an explicitly EA framework when currently only 6.7% of Americans have heard of EA (https://forum.effectivealtruism.org/posts/qQMLGqe4z95i6kJPE/how-many-people-have-heard-of-effective-altruism). This is because there is serious risk of many people’s first impressions of EA to be bad/politicized, with bad consequences for the longterm potential of the movement. This is because political opponents will be incentivized to attack EA directly when attacking a candidate running on an EA platform. If people are exposed to EA in other more faithful ways first, EA is likely to be more successful longterm.
To me it seems the main concern is with using expected value maximization, not with longtermism. Rather than being rationally required to take an action with the highest expected value, I think you are probably only rationally required not to take any action resulting in a world that is worse than an alternative at every percentile of the probability distribution. So in this case you would not have to take the bet because at the 0.1st percentile of the probability distribution taking the bet has a lower value than status quo, while at the 99th percentile it has a higher value.
In practice, this still ends up looking approximately like expected value maximization for most EA decisions because of the huge background uncertainty about what the world will look like. (My current understanding is that you can think of this as an extended version of “if everyone in EA took risky high EV options, then the aggregate result will pretty consistently/with low risk be near the total expected value”)
See this episode of the 80,000 hours podcast for a good description of this “stochastic dominance” framework: https://80000hours.org/podcast/episodes/christian-tarsney-future-bias-fanaticism/.
Yes, I have a group going now!
To what extent are there already similarly dangerous pathogen genomes on the internet? I’m guessing that things like smallpox are less of a worry because we already have a vaccine for them, but if many novel, certified pandemic-grade pathogen genomes are already available then adding more seems significantly less harmful.
I was wondering how Rohin tried starting the group. If he was doing it remotely, then it seems like that may have been a factor in why it failed the second time (because it would be hard to form a community). Thanks for suggesting messaging the people who most recently joined the UW EA Facebook group—I didn’t think there were any new people, but there are a few!
I did get in contact with Jessica McCurdy from CEA, who I plan to talk to soon about getting started and the fall accelerator program, and I just filled out the interest form. I’ll keep looking for EAs on Reddit—thanks for the suggestion! I think it would be valuable to have a chat with you as well.
[Question] Are there any current students at University of Washington?
Hey everyone, I’m also new to the forum and to EA as of summer 2021. I found EA mostly through Lex Fridman’s old podcast with Will MacAskill, which I watched after being reminded of EA by a friend. Then I read some articles on 80,000 hours and was pretty convinced.
I’m a sophomore computer science student at the University of Washington. I’m currently doing research with UW Applied Math on machine learning for science and engineering. It seems like my most likely career is in research in AI or brain-computer interfacing, but I’m still deciding and have an appointment with 80,000 hours advising.
Something else I’m interested in is joining (and possibly building) an EA community at UW. To my knowledge, the group has mostly died away since COVID, but there may still be some remaining UW EAs to link up with.
Looking forward to engaging in discussion on the forum!
I wonder if the probability L = 90% is an overestimate of the likelihood of lasting indefinitely. It seems reasonable that the regime could end because of a global catastrophe that is not existential, reverting us to preindustrial society. For example, nuclear war could end regimes if/while there are multiple states, or climate change could cause massive famine. On the other hand, is it reasonable to think that BCI would create so much stability that even the death of a significant proportion of its populace would not be able to end it?
I’m guessing it would be a good idea to talk to people more skeptical about this project so that you can avoid unilateralist curses. It’s not clear how much you’ve done that already (apart from posting on the forum!).
How long do you expect students to participate?
Too much focus on existing top EA focus areas can lead to community stickiness. If this is just meant as a somewhat quick pipeline to introduce people to the ideas of EA once they’ve already settled into a field this might be okay. Also most EAs historically have been convinced at a younger age (<30) when they are more flexible.