I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
A: incorporate X into M;
B: knowingly maintain M while excluding X.
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don’t want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
Is this a good instrument for achieving my objective?
and
Is this a reliable representation of the reality on which that objective depends?
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
“Perhaps we should omit this new finding, if the consequences of doing so are better?”
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
What makes the model a sufficiently reliable model on which that practical reasoning can be based?
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don’t have to be consequences I have already specified.
That is precisely the point.
If I say:
“I should incorporate X because doing so will produce better consequences,”
then I have accepted the structure of P1.
But my claim is:
“I should not knowingly exclude a relevant aspect of reality merely to preserve my existing representation.”
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don’t know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
A: incorporate X into M;
B: knowingly maintain M while excluding X.
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don’t want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
and
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don’t have to be consequences I have already specified.
That is precisely the point.
If I say:
then I have accepted the structure of P1.
But my claim is:
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don’t know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
That is the transition I am interested in.