Most of our impact comes from acausal effects, and effects on the base reality if we are in a simulation: Iâm confused here like everyone else, but I currently donât buy this as a major factor because we only know our reality, and therefore the same things that are good here should naively also have good acausal effects in expectation.
If I understand correctly, this is the âExtrapolationâ response to unawareness I discuss here. What do you think of my response?
I think itâs not quite your âExtrapolationâ, because itâs specifically about the acausal mechanismâby definition, the only (EDIT: direct) acausal effect possible is to make other agents take similar actions to us. (and then the simulation thing I kind of sweep under the rug because the footnote was quickly written, but the argument is somewhat similar (although very vague and Iâd like something better): Whatever purpose our simulators have for simulating us, itâs probably good for their reality too if we reach a good outcome in the simulation.)
Hmm, these arguments seem too anchored on what we happen to currently be aware of.
I think weâre very confused about the theory of acausal control /â what âsimilarityâ is, and should expect lots of crucial considerations to come up there. (Some relevant writings on this here and here.) As one example: If you try to do something cooperative because this gives evidence (or logically causes) that other agents try to do good things for your values, their attempts to help your values might backfire.
I donât see what the basis is for inferring âitâs probably good for their reality too if we reach a good outcome in the simulationâ. Why would we think we can understand their motivations? Theyâd be alien intelligences.
OK I read the LW comment you linked and skimmed the post, but I donât see how they show that we should expect lots of crucial considerations to come up specificallyâthey seem to argue more for âweâre clueless about how much we should do ECLâ? (but correct me if Iâm wrong, I may have missed something). On your example, but why should I expect their attempts to do so to backfire if I donât already expect our own attempts to backfire? That seems like it just grounds out in the original debate. (btw, I also would like to get less confused about the similarity thing)
On the simulators, it just seems like its hard to think of possible simulator-motivations where us reaching good outcomes in the simulation would be bad for the base reality, and easy to think of ones where it would be neutral or good.
In general, I think the unawareness angle is genuinely interesting, Iâm just not moved by it as much as you, probably for a few different reasons that would take some time to articulate.
they seem to argue more for âweâre clueless about how much we should do ECLâ?
I think they suggest that thereâs just a lot of subtlety in working out the implications of acausal decision theories in practice. Which is reason to expect more crucial considerations in this domain generally /â reason to doubt your âby definitionâ argument.
but why should I expect their attempts to do so to backfire
Why should you expect them to be positive in expectation either? (The broader point of the unawareness sequence is that thereâs an ambiguous pile of positive and negative effects to weigh up.)
On the simulators, it just seems like its hard to think of possible simulator-motivations where us reaching good outcomes in the simulation would be bad for the base reality, and easy to think of ones where it would be neutral or good.
But then weâre back to the Extrapolation argument, which you claimed you werenât committed to. Iâm saying, even if the balance of effects we can think of looks good, weâre looking at a super tiny sliver of the set of effects our fully aware selves would be weighing up â and itâs a biased sample of such effects, so extrapolating from that sample is dubious.
âBut then weâre back to the Extrapolation argument, which you claimed you werenât committed to.â
No, I didnât claim thatâI said the snippet you quoted wasnât the Extrapolation argument, and I stand by that. Iâm definitely sympathetic to something like Extrapolation in general.
The original snippet was more about the acausal stuff, and wasnât Extrapolation, and the distinct argument I subsequently mentioned about simulators was Extrapolation.
Thereâs no quick objection I can give to your response specifically (except to simplistically say âno, I donât buy it, we know more than that, we can have some reasonable guesses about simulatorsâ intentionsâ) - properly laying out my disagreements would take a little bit of time and effort, as is usually the case for deep worldview differences.
If I understand correctly, this is the âExtrapolationâ response to unawareness I discuss here. What do you think of my response?
I think itâs not quite your âExtrapolationâ, because itâs specifically about the acausal mechanismâby definition, the only (EDIT: direct) acausal effect possible is to make other agents take similar actions to us.
(and then the simulation thing I kind of sweep under the rug because the footnote was quickly written, but the argument is somewhat similar (although very vague and Iâd like something better): Whatever purpose our simulators have for simulating us, itâs probably good for their reality too if we reach a good outcome in the simulation.)
Hmm, these arguments seem too anchored on what we happen to currently be aware of.
I think weâre very confused about the theory of acausal control /â what âsimilarityâ is, and should expect lots of crucial considerations to come up there. (Some relevant writings on this here and here.) As one example: If you try to do something cooperative because this gives evidence (or logically causes) that other agents try to do good things for your values, their attempts to help your values might backfire.
I donât see what the basis is for inferring âitâs probably good for their reality too if we reach a good outcome in the simulationâ. Why would we think we can understand their motivations? Theyâd be alien intelligences.
OK I read the LW comment you linked and skimmed the post, but I donât see how they show that we should expect lots of crucial considerations to come up specificallyâthey seem to argue more for âweâre clueless about how much we should do ECLâ? (but correct me if Iâm wrong, I may have missed something). On your example, but why should I expect their attempts to do so to backfire if I donât already expect our own attempts to backfire? That seems like it just grounds out in the original debate.
(btw, I also would like to get less confused about the similarity thing)
On the simulators, it just seems like its hard to think of possible simulator-motivations where us reaching good outcomes in the simulation would be bad for the base reality, and easy to think of ones where it would be neutral or good.
In general, I think the unawareness angle is genuinely interesting, Iâm just not moved by it as much as you, probably for a few different reasons that would take some time to articulate.
I think they suggest that thereâs just a lot of subtlety in working out the implications of acausal decision theories in practice. Which is reason to expect more crucial considerations in this domain generally /â reason to doubt your âby definitionâ argument.
Why should you expect them to be positive in expectation either? (The broader point of the unawareness sequence is that thereâs an ambiguous pile of positive and negative effects to weigh up.)
But then weâre back to the Extrapolation argument, which you claimed you werenât committed to. Iâm saying, even if the balance of effects we can think of looks good, weâre looking at a super tiny sliver of the set of effects our fully aware selves would be weighing up â and itâs a biased sample of such effects, so extrapolating from that sample is dubious.
âBut then weâre back to the Extrapolation argument, which you claimed you werenât committed to.â
No, I didnât claim thatâI said the snippet you quoted wasnât the Extrapolation argument, and I stand by that. Iâm definitely sympathetic to something like Extrapolation in general.
(Edited for tone)
Sorry, I donât understand. The snippet I quoted â about acausal stuff and simulations â is whatâs at issue in this discussion.
Regardless, Iâm still interested in where you object to my response to Extrapolation. Could you please say more on that?
The original snippet was more about the acausal stuff, and wasnât Extrapolation, and the distinct argument I subsequently mentioned about simulators was Extrapolation.
Thereâs no quick objection I can give to your response specifically (except to simplistically say âno, I donât buy it, we know more than that, we can have some reasonable guesses about simulatorsâ intentionsâ) - properly laying out my disagreements would take a little bit of time and effort, as is usually the case for deep worldview differences.