What do you think about participating in a forecasting platform, e.g. Good Judgement Open or Metaculus? It seems to cover all ingredients, and even be a good signal for others to evaluate your judgement quality.
Seems pretty good for predicting things about the world that get resolved on short timescales. Sadly it seems less helpful for practicing judgement about things like the following:
judging arguments about things like the moral importance of wild animal suffering, plausibility of AI existential risk, and existence of mental illness
long-term predictions
predictions about small-scale things like how a project should be organized (though you can train calibration on this kind of question)
Re my own judgement: I appreciate your confidence in me. I spend a lot of time talking to people who have IMO better judgement than me; most of the things I say in this post (and a reasonable chunk of things I say other places) are my rephrasings of their ideas. I think that people whose judgement I trust would agree with my assessment of my judgement quality as “good in some ways” (this was the assessment of one person I asked about this in response to your comment).
Seems pretty good for predicting things about the world that get resolved on short timescales. Sadly it seems less helpful for practicing judgement about things like the following:
judging arguments about things like the moral importance of wild animal suffering, plausibility of AI existential risk, and existence of mental illness
long-term predictions
predictions about small-scale things like how a project should be organized (though you can train calibration on this kind of question)
Re my own judgement: I appreciate your confidence in me. I spend a lot of time talking to people who have IMO better judgement than me; most of the things I say in this post (and a reasonable chunk of things I say other places) are my rephrasings of their ideas. I think that people whose judgement I trust would agree with my assessment of my judgement quality as “good in some ways” (this was the assessment of one person I asked about this in response to your comment).