My preference also does not depend on whether I am ultimately correct about what I learned. Nor does it depend on predicting that adding X will improve future decisions. Once I judge X to be a relevant part of reality,[1] knowingly maintaining a model that excludes it is defective. No forecast is required.
Defective how? Defective to what end? Perhaps we should omit this new finding, if the consequences of doing so are better?
Moreover, B does not merely omit one effect from one calculation. It preserves a defect in the instrument that produces future assessments. For an impartial altruist, this is not merely an epistemic failure. It is a failure to remain responsive to consequences one already recognises as relevant.
And what consequences are those, that the impartial altruist recognizes as relevant?
I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
A: incorporate X into M;
B: knowingly maintain M while excluding X.
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don’t want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
Is this a good instrument for achieving my objective?
and
Is this a reliable representation of the reality on which that objective depends?
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
“Perhaps we should omit this new finding, if the consequences of doing so are better?”
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
What makes the model a sufficiently reliable model on which that practical reasoning can be based?
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don’t have to be consequences I have already specified.
That is precisely the point.
If I say:
“I should incorporate X because doing so will produce better consequences,”
then I have accepted the structure of P1.
But my claim is:
“I should not knowingly exclude a relevant aspect of reality merely to preserve my existing representation.”
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don’t know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
Defective how? Defective to what end? Perhaps we should omit this new finding, if the consequences of doing so are better?
And what consequences are those, that the impartial altruist recognizes as relevant?
I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
A: incorporate X into M;
B: knowingly maintain M while excluding X.
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don’t want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
and
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don’t have to be consequences I have already specified.
That is precisely the point.
If I say:
then I have accepted the structure of P1.
But my claim is:
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don’t know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
That is the transition I am interested in.