I also would be curious to see some Google Docs about longtermist strategy, as I’ve only ever seen whiteboard drawings at various conferences/events (and maybe a few other written ideas in various places).
However, I’m a bit confused by this:
But I also think there’s many epistemic and accountability risks endemic to long-termism, such as how it’s easy to pitch for results you’ll (probably) never be around to see or be accountable for; I notice this thinking flaw in myself when I think about long-term interventions.
I may not be fully understanding your hesitance here, but if your point is saying something like “you may not be held accountable for your long-term impacts and thus are more likely to make mistakes/be biased,” I feel like you’re potentially really overweighting this issue:
How often do people get held directly/accurately accountable for their near-term actions even from a near-term lens? People can write bad-but-persuasive arguments for short-term actions and reap status or other rewards. This is especially problematic when the difficulty of verifying the quality of analysis is really high (e.g., when you can’t run RCTs or rely on engineering models/simulations). Longtermism’s problem here doesn’t seem that unique.
Near-term interventions can also prove to be relatively unimportant from a long-term lens (e.g., saving lives from malaria but those people or their children die anyway in 60 years due to some x-risk that you neglected). Who holds these near-term interventions accountable in the long-term?
You can definitely be held accountable or feel guilty if it becomes apparent in the near-term that your arguments/proposals will actually be bad in the long-term. I don’t particularly support their claims, but people like Kerry Vaughan have been attacking OpenPhil/EA for funding OpenAI due Kerry’s perceptions that it was a bad choice.
Long-termism can probably still just bite the bullet here even if you mostly dismiss the previous points: sure, it might theoretically be harder to be confident that what you are doing is good for the long-term specifically due to this “lack of accountability” argument, but even if that cuts your expected value by 50%, 75%, or even 90%, the expected value of x-risk reduction is still incredibly massive. So, maybe you think “After accounting for my ‘lack-of-accountability’ biases, this action actually only has an expected net effect of 0.001% x-risk reduction (rather than 0.01% as I initially thought),” but that expected value would still be massive.
Agree there’s something to your 1-3 counterarguments but I find the fourth less convincing, maybe more because of semantics than actual substantive disagreement. Why? A difference in net effect of x-risk reduction of 0.01% Vs. 0.001% is pretty massive. These especially matter if the expected value is massive, because sometimes the same expected value holds across multiple areas. For example, preventing asteroid related X risk Vs. Ai Vs. Bio Vs. runaway climate change; (by definition) all same EV (if you take the arguments at face value). But plausibility of each approach and individual interventions within that would be pretty high variance.
A difference in net effect of x-risk reduction of 0.01% Vs. 0.001% is pretty massive. These especially matter if the expected value is massive, because sometimes the same expected value holds across multiple areas. For example, preventing asteroid related X risk Vs. Ai Vs. Bio Vs. runaway climate change; (by definition) all same EV (if you take the arguments at face value). But plausibility of each approach and individual interventions within that would be pretty high variance.
I’m a bit uncertain as to what you are arguing/disputing here. To clarify on my end, my 4th point was mainly just saying “when comparing long-termist vs. near-termist causes, the concern over ‘epistemic and accountability risks endemic to long-termism’ seems relatively unimportant given [my previous 3 points and/or] the orders of magnitude of difference in expected value between near-termism vs. long-termism.”
Your new comment seems to be saying that an order-of-magnitude uncertainty factor is important when comparing cause areas within long-termism, rather than when comparing between overall long-termism and overall near-termism. I will briefly respond to that claim in the next paragraph, but if your new comment is actually still arguing your original point that the potential for bias is concerning enough that it makes the expected value of long-termism less than or just roughly equal to that of near-termism, I’m confused how you came to that conclusion. Could you clarify which argument you are now trying to make?
Regarding the inter-long-termism comparisons, I’ll just say one thing for now: some cause areas still seem significantly less important than other areas. For example, it might make sense for you to focus on x-risks from asteroids or other cosmic events if you have decades of experience in astrophysics and the field is currently undersaturated (although if you are a talented scientist it might make sense for you to offer some intellectual support to AI, bio, or even climate). However, the x-risk from asteroids is many orders of magnitude smaller than that from AI and probably even biological threats. Thus, even an uncertainty factor that for some reason only reduces your estimate of expected x-risk reduction via AI or bio work by a factor of 10 (e.g., from 0.001% to 0.0001%) without also affecting your estimate of expected x-risk reduction via work on asteroid safety will probably not have much effect on the direction of the inequality (i.e., x > y; 0.1x > y).
I also would be curious to see some Google Docs about longtermist strategy, as I’ve only ever seen whiteboard drawings at various conferences/events (and maybe a few other written ideas in various places).
However, I’m a bit confused by this:
I may not be fully understanding your hesitance here, but if your point is saying something like “you may not be held accountable for your long-term impacts and thus are more likely to make mistakes/be biased,” I feel like you’re potentially really overweighting this issue:
How often do people get held directly/accurately accountable for their near-term actions even from a near-term lens? People can write bad-but-persuasive arguments for short-term actions and reap status or other rewards. This is especially problematic when the difficulty of verifying the quality of analysis is really high (e.g., when you can’t run RCTs or rely on engineering models/simulations). Longtermism’s problem here doesn’t seem that unique.
Near-term interventions can also prove to be relatively unimportant from a long-term lens (e.g., saving lives from malaria but those people or their children die anyway in 60 years due to some x-risk that you neglected). Who holds these near-term interventions accountable in the long-term?
You can definitely be held accountable or feel guilty if it becomes apparent in the near-term that your arguments/proposals will actually be bad in the long-term. I don’t particularly support their claims, but people like Kerry Vaughan have been attacking OpenPhil/EA for funding OpenAI due Kerry’s perceptions that it was a bad choice.
Long-termism can probably still just bite the bullet here even if you mostly dismiss the previous points: sure, it might theoretically be harder to be confident that what you are doing is good for the long-term specifically due to this “lack of accountability” argument, but even if that cuts your expected value by 50%, 75%, or even 90%, the expected value of x-risk reduction is still incredibly massive. So, maybe you think “After accounting for my ‘lack-of-accountability’ biases, this action actually only has an expected net effect of 0.001% x-risk reduction (rather than 0.01% as I initially thought),” but that expected value would still be massive.
Agree there’s something to your 1-3 counterarguments but I find the fourth less convincing, maybe more because of semantics than actual substantive disagreement. Why? A difference in net effect of x-risk reduction of 0.01% Vs. 0.001% is pretty massive. These especially matter if the expected value is massive, because sometimes the same expected value holds across multiple areas. For example, preventing asteroid related X risk Vs. Ai Vs. Bio Vs. runaway climate change; (by definition) all same EV (if you take the arguments at face value). But plausibility of each approach and individual interventions within that would be pretty high variance.
I’m a bit uncertain as to what you are arguing/disputing here. To clarify on my end, my 4th point was mainly just saying “when comparing long-termist vs. near-termist causes, the concern over ‘epistemic and accountability risks endemic to long-termism’ seems relatively unimportant given [my previous 3 points and/or] the orders of magnitude of difference in expected value between near-termism vs. long-termism.”
Your new comment seems to be saying that an order-of-magnitude uncertainty factor is important when comparing cause areas within long-termism, rather than when comparing between overall long-termism and overall near-termism. I will briefly respond to that claim in the next paragraph, but if your new comment is actually still arguing your original point that the potential for bias is concerning enough that it makes the expected value of long-termism less than or just roughly equal to that of near-termism, I’m confused how you came to that conclusion. Could you clarify which argument you are now trying to make?
Regarding the inter-long-termism comparisons, I’ll just say one thing for now: some cause areas still seem significantly less important than other areas. For example, it might make sense for you to focus on x-risks from asteroids or other cosmic events if you have decades of experience in astrophysics and the field is currently undersaturated (although if you are a talented scientist it might make sense for you to offer some intellectual support to AI, bio, or even climate). However, the x-risk from asteroids is many orders of magnitude smaller than that from AI and probably even biological threats. Thus, even an uncertainty factor that for some reason only reduces your estimate of expected x-risk reduction via AI or bio work by a factor of 10 (e.g., from 0.001% to 0.0001%) without also affecting your estimate of expected x-risk reduction via work on asteroid safety will probably not have much effect on the direction of the inequality (i.e., x > y; 0.1x > y).