Hi! I’m a twenty three year old ex-software engineer. I like to write about EA and I’m interested in a broad range of its related topics (such as animal welfare, moral progress, and biorisks.)
My YouTube channel is James Brobin, and it conveys EA ideas to a general audience.
My blog is James Brobin’s Substack, and it includes stuff that I haven’t posted on this forum.
Yeah, to be fair, I do think that the broad trend argument is probably the strongest argument for longtermism. Like, it appears that humanity has continuously expanded and grown for the past 12,000 years. If this trend continues, then it makes sense to think that eventually we’ll have the technology to start expanding into space.
Ah, thank you!
I’m glad you looked at it!
Thanks!
Yeah, I agree that it’s important to get more information than merely “fear of pandemic.”
If you haven’t already, I’d recommend reading Abi Olvera’s series on this. She argues that AI people think about virology completely differently than virology people do. From the perspective she offers, engineered pandemics seem a lot less concerning than the AI community makes them out to be.
The metabolic energy argument makes sense. It’s unclear to me whether we should apply any kind of power law though since consciousness seems to be different in nature, as I view it as likely non-physical.
Three things:
I don’t think we should take the enormous size of the future into account because I don’t think we have sufficient knowledge about the future to have any good guesses on how large humanity will become or whether it will create a world that we would approve of. As such, I would measure the value of existential risk reduction much more conservatively, such as the number of humans over the next two hundred years who wouldn’t be born.
I find the chance of an existential risk from biorisks to be extremely low (I.e. 10^-7). This is because I think that, for a biorisk to be an existential risk, I think that it would have to make the surface of the Earth unlivable for an extended period of time. Otherwise, a significant number of humans will survive and be able to continue civilization. I find this unlikely to occur because, even if this is technically possible, it would be extraordinarily difficult. (That said, I haven’t looked into mirror bacteria.)
Even among people who are worried about engineered pandemics, virtually none of the are worried about existential risk. As such, if we’re basing our view on other people’s view, and (1) very few experts are concerned about engineered pandemics (2) very few of those experts are worried about existential risk (3) we have the prior that we should significantly discount fringe views because they’re almost always wrong, then we can reasonably get to quite a low risk of engineered pandemics.
I definitely agree with the idea of doing a bit of a numbers game though. If the numbers suggest we should be really focused on biorisks over other concerns, I think it makes sense to allocate a lot of resources towards it.
Thanks for sharing that post. I made this comment on it in case you’re curious.
You write:
Generating higher-intensity affective states likely involves greater activation within nociceptive pathways and broader engagement of central processing systems. In both vertebrates and invertebrates, increasing signal strength is associated with recruiting additional processing units and enhanced synaptic transmission, which demand greater energy
This seems plausible to me.
Then, you argue:
Measuring the energetic and computational demands of pain-processing systems across species could help determine whether primitive nervous systems had the capacity to support high-intensity pain.
One problem I find with this is that it seems like it would be very difficult to assess how much “energetic and computational demands” constitutes high-intensity pain.
For instance, let’s imagine that, hypothetically, each organism has one part of its nervous system where the amount of energy that runs through it is linearly related to the total amount of pain the organism experience.
If this were the case, we could then measure how much energy corresponds to high-intensity pain in humans. Then, we could find primitive organisms and measure how much energy goes through their neuronal pain region. If it were the same amount as humans for high-intensity pain, then we would know that they also experience high-intensity pain.
Problematically, though, this hypothetical scenario is very different from reality:
We don’t know how whether energy linearly corresponds to pain.
The amount of activation in the human brain doesn’t correspond to the amount of pain experienced.
We have not mapped out what exact parts of the human brain cause pain and how different patterns of activation correspond to different pain levels.
That said, it seems like this framework may help us to compare pain across simple organisms though. If two organisms are quite similar but one uses much more energy, then, perhaps, it experiences much more pain.
You write:
Evolutionarily speaking, nothing would change if the rewards and punishment had a value of “alternative value” = k*”value”, where k is positive? The expected value of each action would have been k times as large, but the ratio between the expected value of any 2 actions would have been exactly the same. k can in principle be arbitrarily close to 0. So experiences could have an intensity arbitrarily close to 0 while leading to exactly the same behaviour?
This question gets to me a lot. It seems totally unclear how much the valence of an experience correlates to behavior. Like, it seems plausible that extraordinarily little pain could motivate behavior in simple organisms, but it’s also hard to imagine, since, as a complex organism, I need pretty intense pain to be motivated to significantly change my behavior. And, it doesn’t seem like there’s any way we could reduce uncertainty about this.
--
Also, in regards to Bob’s book, it seems like it would make sense, rather than adding the model’s estimates together to instead act on each model separately, i.e. assigning 5% of resources to help animals according to the equality model, 25% according to the neurophysiological model, and 70% according to the proxies model. Then, within the 70% for the proxies model, we could assign 33% of resources to applying a high exponent to the welfare ranges, 33% to applying no exponent to the welfare ranges, and 33% to applying a low exponent to the welfare ranges.
The problem of what percentages you should use and what models to use would, of course, still remain though.
Yeah, I agree. I think I should have framed it as like ways in which Bayesian thinking fails rather than as an attack on Bayesian reasoning.
Thanks! I really like that example you gave about asking seven year olds. This is definitely a major criticism of mine of Bob Fischer’s book. It seems like different experts could easily have come to very different conclusions because of what models they choose or, as you point out, how they weigh them.
I was making in-paradigm critiques, since, if something is internally invalid, it gives us good reason to think it is also externally invalid.
I linked to the XPT tournament just as an example of asking non-experts for predictions. I thought that pandemics were a good example since most people don’t know much about pandemics at all so we should expect their guesses to be very off. I could see the argument that forecasting research has found forecasters to be better than experts so it makes sense to ask non-experts, but it’s important to note that we haven’t validated this over very long time periods.
I linked to the GPI paper since it seems to pretty absurd to me to try to guess how many future there will be. Even if we can come up with accurate estimates for carrying capacity for different regions of space, I have no clue how we could predict the likelihood of reaching carrying capacity in these regions of space.
In regards to your last point, I think my view is most basically that, by assigning probabilities to outcomes, we’re giving ourselves excessive confidence when we often have too little knowledge to warrant the confidence, that correct decision theories should take into account our cluelessness to a much greater extent, and that people probably do Bayesian reasoning a lot worse than they think.
That said, I’m still figuring out my views on decisions theories so I probably should have researched this topic a lot more before making a post.
For your last point, I agree with you that we run into the problem of EV-maxxing being undefined, but I don’t know where to go from there. It doesn’t seem to make sense to me to do anything in regards to something with such a wide probability range because it seems like you’ll just spend all your time chasing things that you know very little about but which suggest really high EV.
Thanks for the thoughtful comment.
Thanks for explaining that!
Yeah, I think that articulates a lot of my criticism of it. It seems like people generally aren’t very good at making guesses about probabilities of things in general so we shouldn’t expect that we can engage in “Bayesianism” in our head and magically create really good predictions.
I also wonder about this. I’m excited to see where research in the field leads.