This was really great. As someone who has been lurking around LW/EA Forum for a few years but has never found reading the Sequences the highest-return investment compared to other things I could be doing, I very much appreciate your writing it.
A thought on something which is probably not core to your post but worth considering:
You said:
The dream behind the Bayesian mindset is that I could choose some set of values that I can really stand behind (e.g., putting a lot of value on helping people, and none on things like “feeling good about myself”), and focus only on that. Then the parts of myself driven by “bad” values would have to either quiet down, or start giving non-sincere probabilities. But over time, I could watch how accurate my probabilities are, and learn to listen to the parts of myself that make better predictions.
I think it’s perhaps… not feasible, or has long-term side effects, to think that if you currently care about feeling good about yourself, you can just decide you don’t and jump immediately to ignoring that need of yours. I would predict that taking that approach is likely to result in resistance to making accurate predictions or doing the things you endorse valuing, and/ or mysterious unhappy emotions because your need to feel good about yourself is not being met.
It seems to me that it would be better to use some method like Internal Double Crux to dialogue between the part of you that wants to generate good feelings by generating skewed predictions and the part of you that wants to help people, and find a way to meet the former part’s needs that don’t require making skewed predictions. An example of such an approach could be feeling good about yourself for cultivating more effective predictions. I imagine that’s implicit in the approach you describe, but it may be more effective to hold explicit space for the part that wants to feel good about itself, rather than making it wrong for generating skewed predictions.
+1. That paragraph to me reads like: “Here’s a neat trick by which you can forcibly self-modify to care about fewer, simpler, easier-to-measure things! Yay!”
I agree with what you say here, as a general matter. I’m not sure I want to make an edit, as I really do think there are some “bad” parts of myself that I’d prefer to “expose and downweight,” but I agree that it’s easy to get carried away with that sort of thing.
This was really great. As someone who has been lurking around LW/EA Forum for a few years but has never found reading the Sequences the highest-return investment compared to other things I could be doing, I very much appreciate your writing it.
A thought on something which is probably not core to your post but worth considering:
You said:
I think it’s perhaps… not feasible, or has long-term side effects, to think that if you currently care about feeling good about yourself, you can just decide you don’t and jump immediately to ignoring that need of yours. I would predict that taking that approach is likely to result in resistance to making accurate predictions or doing the things you endorse valuing, and/ or mysterious unhappy emotions because your need to feel good about yourself is not being met.
It seems to me that it would be better to use some method like Internal Double Crux to dialogue between the part of you that wants to generate good feelings by generating skewed predictions and the part of you that wants to help people, and find a way to meet the former part’s needs that don’t require making skewed predictions. An example of such an approach could be feeling good about yourself for cultivating more effective predictions. I imagine that’s implicit in the approach you describe, but it may be more effective to hold explicit space for the part that wants to feel good about itself, rather than making it wrong for generating skewed predictions.
+1. That paragraph to me reads like: “Here’s a neat trick by which you can forcibly self-modify to care about fewer, simpler, easier-to-measure things! Yay!”
I agree with what you say here, as a general matter. I’m not sure I want to make an edit, as I really do think there are some “bad” parts of myself that I’d prefer to “expose and downweight,” but I agree that it’s easy to get carried away with that sort of thing.
Cool. We are on the same page then. :) I also agree that there are some bad parts of myself I’d prefer to expose and downweight.